{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating Keras DNN model\n",
    "\n",
    "**Learning Objectives**\n",
    "\n",
    "1. Create input layers for raw features\n",
    "1. Create feature columns for inputs\n",
    "1. Create DNN dense hidden layers and output layer\n",
    "1. Build DNN model tying all of the pieces together\n",
    "1. Train and evaluate\n",
    "\n",
    "\n",
    "## Introduction \n",
    "In this notebook, we'll be using Keras to create a DNN model to predict the weight of a baby before it is born.\n",
    "\n",
    "We'll start by defining the CSV column names, label column, and column defaults for our data inputs. Then, we'll construct a tf.data Dataset of features and the label from the CSV files and create inputs layers for the raw features. Next, we'll set up feature columns for the model inputs and build a deep neural network in Keras. We'll create a custom evaluation metric and build our DNN model. Finally, we'll train and evaluate our model.\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__ in this student lab notebook -- try to complete this notebook first and then review the [solution notebook](https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/courses/machine_learning/deepdive2/end_to_end_ml/solutions/keras_dnn_babyweight.ipynb)."
     ]
  },
  {
     "cell_type": "markdown",
     "metadata": {
      "colab_type": "text",
      "id": "hJ7ByvoXzpVI"
     },
     "source": [
      "## Set up environment variables and load necessary libraries"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": null,
     "metadata": {},
     "outputs": [],
     "source": [
      "!sudo chown -R jupyter:jupyter /home/jupyter/training-data-analyst"
     ]
    },
    {
  "cell_type": "code",
  "execution_count": null,
  "metadata": {},
  "outputs": [
   {
    "name": "stdout",
    "output_type": "stream",
    "text": [
"Collecting google-cloud-bigquery==1.25.0\n",
"Downloading google_cloud_bigquery-1.25.0-py2.py3-none-any.whl (169 kB)\n",
    "|████████████████████████████████| 169 kB 4.7 MB/s eta 0:00:01\n",

"Requirement already satisfied:  six in /home/jupyter/.local/lib/python3.7/site-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: google-auth in /usr/local/lib/python3.7/site-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: google-resumable-media in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: google-cloud-core in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: protobuf in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: google-api-core in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: cachetools in /usr/local/lib/python3.7/dist-packages(from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: rsa in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: pyasn1-modules in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: googleapis-common-protos in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: pyasn1 in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from google-cloud-bigquery==1.25.0)\n",
"Installing collected packages: google-resumable-media, google-cloud-bigquery\n",
"\u001b[33mWARNING: You are using pip version 20.1; however, version 20.2.3 is available."
    ]
   }
  ],
  "source": [
   "!pip install --user google-cloud-bigquery==3.4.1"
  ]
 },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "**Note**: Restart your kernel to use updated packages."
     ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Kindly ignore the deprecation warnings and incompatibility errors related to google-cloud-storage."
      ]
     },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Import necessary libraries."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 1,
     "metadata": {},
     "outputs": [],
     "source": [
      "from google.cloud import bigquery\n",
      "import pandas as pd\n",
      "import datetime\n",
      "import os\n",
      "import shutil\n",
      "import matplotlib.pyplot as plt\n",
      "import tensorflow as tf\n",
      "print(tf.__version__)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Set environment variables so that we can use them throughout the notebook."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": null,
     "metadata": {},
     "outputs": [],
     "source": [
      "%%bash\n",
      "export PROJECT=$(gcloud config list project --format \"value(core.project)\")\n",
      "echo \"Your current GCP Project Name is: \"$PROJECT"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 3,
     "metadata": {},
     "outputs": [],
     "source": [
      "PROJECT = \"cloud-training-demos\"  # Replace with your PROJECT"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Create ML datasets by sampling using BigQuery\n",
      "\n",
      "We'll begin by sampling the BigQuery data to create smaller datasets. Let's create a BigQuery client that we'll use throughout the lab."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 4,
     "metadata": {},
     "outputs": [],
     "source": [
      "bq = bigquery.Client(project = PROJECT)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We need to figure out the right way to divide our hash values to get our desired splits. To do that we need to define some values to hash within the module. Feel free to play around with these values to get the perfect combination."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 5,
     "metadata": {},
     "outputs": [],
     "source": [
      "modulo_divisor = 100\n",
      "train_percent = 80.0\n",
      "eval_percent = 10.0\n",
      "\n",
      "train_buckets = int(modulo_divisor * train_percent / 100.0)\n",
      "eval_buckets = int(modulo_divisor * eval_percent / 100.0)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can make a series of queries to check if our bucketing values result in the correct sizes of each of our dataset splits and then adjust accordingly. Therefore, to make our code more compact and reusable, let's define a function to return the head of a dataframe produced from our queries up to a certain number of rows."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 6,
     "metadata": {},
     "outputs": [],
     "source": [
      "def display_dataframe_head_from_query(query, count=10):\n",
      "    \"\"\"Displays count rows from dataframe head from query.\n",
      "    \n",
      "    Args:\n",
      "        query: str, query to be run on BigQuery, results stored in dataframe.\n",
      "        count: int, number of results from head of dataframe to display.\n",
      "    Returns:\n",
      "        Dataframe head with count number of results.\n",
      "    \"\"\"\n",
      "    df = bq.query(\n",
      "        query + \" LIMIT {limit}\".format(\n",
      "            limit=count)).to_dataframe()\n",
      "\n",
      "    return df.head(count)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "For our first query, we're going to use the original query above to get our label, features, and columns to combine into our hash which we will use to perform our repeatable splitting. There are only a limited number of years, months, days, and states in the dataset. Let's see what the hash values are. We will need to include all of these extra columns to hash on to get a fairly uniform spread of the data. Feel free to try less or more in the hash and see how it changes your results."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 7,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>is_male</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>year</th>\n",
         "      <th>month</th>\n",
         "      <th>date</th>\n",
         "      <th>state</th>\n",
         "      <th>mother_birth_state</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>7.568469</td>\n",
         "      <td>True</td>\n",
         "      <td>22</td>\n",
         "      <td>1</td>\n",
         "      <td>46</td>\n",
         "      <td>2001</td>\n",
         "      <td>7</td>\n",
         "      <td>5</td>\n",
         "      <td>CA</td>\n",
         "      <td>CA</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>8.807467</td>\n",
         "      <td>True</td>\n",
         "      <td>39</td>\n",
         "      <td>1</td>\n",
         "      <td>42</td>\n",
         "      <td>2001</td>\n",
         "      <td>8</td>\n",
         "      <td>3</td>\n",
         "      <td>CA</td>\n",
         "      <td>Foreign</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>8.313632</td>\n",
         "      <td>True</td>\n",
         "      <td>23</td>\n",
         "      <td>1</td>\n",
         "      <td>35</td>\n",
         "      <td>2001</td>\n",
         "      <td>10</td>\n",
         "      <td>7</td>\n",
         "      <td>IL</td>\n",
         "      <td>IL</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>8.000575</td>\n",
         "      <td>False</td>\n",
         "      <td>27</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>2001</td>\n",
         "      <td>6</td>\n",
         "      <td>7</td>\n",
         "      <td>IL</td>\n",
         "      <td>IL</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>6.563162</td>\n",
         "      <td>False</td>\n",
         "      <td>29</td>\n",
         "      <td>1</td>\n",
         "      <td>39</td>\n",
         "      <td>2001</td>\n",
         "      <td>11</td>\n",
         "      <td>7</td>\n",
         "      <td>KY</td>\n",
         "      <td>IN</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>7.125340</td>\n",
         "      <td>False</td>\n",
         "      <td>34</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>2001</td>\n",
         "      <td>12</td>\n",
         "      <td>7</td>\n",
         "      <td>MD</td>\n",
         "      <td>MD</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>7.438397</td>\n",
         "      <td>False</td>\n",
         "      <td>31</td>\n",
         "      <td>1</td>\n",
         "      <td>38</td>\n",
         "      <td>2001</td>\n",
         "      <td>4</td>\n",
         "      <td>3</td>\n",
         "      <td>MA</td>\n",
         "      <td>Foreign</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>7.352416</td>\n",
         "      <td>True</td>\n",
         "      <td>30</td>\n",
         "      <td>1</td>\n",
         "      <td>37</td>\n",
         "      <td>2001</td>\n",
         "      <td>5</td>\n",
         "      <td>7</td>\n",
         "      <td>MI</td>\n",
         "      <td>MI</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>8.062305</td>\n",
         "      <td>True</td>\n",
         "      <td>16</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>2001</td>\n",
         "      <td>10</td>\n",
         "      <td>5</td>\n",
         "      <td>MN</td>\n",
         "      <td>MN</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>7.251004</td>\n",
         "      <td>True</td>\n",
         "      <td>17</td>\n",
         "      <td>1</td>\n",
         "      <td>39</td>\n",
         "      <td>2001</td>\n",
         "      <td>2</td>\n",
         "      <td>5</td>\n",
         "      <td>MS</td>\n",
         "      <td>MS</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   weight_pounds  is_male  mother_age  plurality  gestation_weeks  year  \\\n",
         "0       7.568469     True          22          1               46  2001   \n",
         "1       8.807467     True          39          1               42  2001   \n",
         "2       8.313632     True          23          1               35  2001   \n",
         "3       8.000575    False          27          1               40  2001   \n",
         "4       6.563162    False          29          1               39  2001   \n",
         "5       7.125340    False          34          1               40  2001   \n",
         "6       7.438397    False          31          1               38  2001   \n",
         "7       7.352416     True          30          1               37  2001   \n",
         "8       8.062305     True          16          1               40  2001   \n",
         "9       7.251004     True          17          1               39  2001   \n",
         "\n",
         "   month  date state mother_birth_state  \n",
         "0      7     5    CA                 CA  \n",
         "1      8     3    CA            Foreign  \n",
         "2     10     7    IL                 IL  \n",
         "3      6     7    IL                 IL  \n",
         "4     11     7    KY                 IN  \n",
         "5     12     7    MD                 MD  \n",
         "6      4     3    MA            Foreign  \n",
         "7      5     7    MI                 MI  \n",
         "8     10     5    MN                 MN  \n",
         "9      2     5    MS                 MS  "
        ]
       },
       "execution_count": 7,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Get label, features, and columns to hash and split into buckets\n",
      "hash_cols_fixed_query = \"\"\"\n",
      "SELECT\n",
      "    weight_pounds,\n",
      "    is_male,\n",
      "    mother_age,\n",
      "    plurality,\n",
      "    gestation_weeks,\n",
      "    year,\n",
      "    month,\n",
      "    CASE\n",
      "        WHEN day IS NULL THEN\n",
      "            CASE\n",
      "                WHEN wday IS NULL THEN 0\n",
      "                ELSE wday\n",
      "            END\n",
      "        ELSE day\n",
      "    END AS date,\n",
      "    IFNULL(state, \"Unknown\") AS state,\n",
      "    IFNULL(mother_birth_state, \"Unknown\") AS mother_birth_state\n",
      "FROM\n",
      "    publicdata.samples.natality\n",
      "WHERE\n",
      "    year > 2000\n",
      "    AND weight_pounds > 0\n",
      "    AND mother_age > 0\n",
      "    AND plurality > 0\n",
      "    AND gestation_weeks > 0\n",
      "\"\"\"\n",
      "\n",
      "display_dataframe_head_from_query(hash_cols_fixed_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using `COALESCE` would provide the same result as the nested `CASE WHEN`. This is preferable when all we want is the first non-null instance. To be precise the `CASE WHEN` would become `COALESCE(wday, day, 0) AS date`. You can read more about it [here](https://cloud.google.com/bigquery/docs/reference/standard-sql/conditional_expressions)."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Next query will combine our hash columns and will leave us just with our label, features, and our hash values."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 8,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>is_male</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>7.109908</td>\n",
         "      <td>False</td>\n",
         "      <td>25</td>\n",
         "      <td>1</td>\n",
         "      <td>38</td>\n",
         "      <td>563561248331884029</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>7.588311</td>\n",
         "      <td>False</td>\n",
         "      <td>19</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>3487851893553562338</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>4.812691</td>\n",
         "      <td>True</td>\n",
         "      <td>35</td>\n",
         "      <td>1</td>\n",
         "      <td>33</td>\n",
         "      <td>2669304657201106008</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>7.251004</td>\n",
         "      <td>True</td>\n",
         "      <td>30</td>\n",
         "      <td>2</td>\n",
         "      <td>38</td>\n",
         "      <td>7076342771382320241</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>6.206013</td>\n",
         "      <td>False</td>\n",
         "      <td>21</td>\n",
         "      <td>1</td>\n",
         "      <td>36</td>\n",
         "      <td>8828960867056723893</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>6.062712</td>\n",
         "      <td>False</td>\n",
         "      <td>33</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>4280252324912833683</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>7.500126</td>\n",
         "      <td>False</td>\n",
         "      <td>19</td>\n",
         "      <td>1</td>\n",
         "      <td>39</td>\n",
         "      <td>6090508671071281093</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>7.687519</td>\n",
         "      <td>True</td>\n",
         "      <td>23</td>\n",
         "      <td>1</td>\n",
         "      <td>41</td>\n",
         "      <td>8708360030053768340</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>8.875811</td>\n",
         "      <td>True</td>\n",
         "      <td>24</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>8530116731648975419</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>7.387690</td>\n",
         "      <td>False</td>\n",
         "      <td>28</td>\n",
         "      <td>1</td>\n",
         "      <td>38</td>\n",
         "      <td>1776323475383399588</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   weight_pounds  is_male  mother_age  plurality  gestation_weeks  \\\n",
         "0       7.109908    False          25          1               38   \n",
         "1       7.588311    False          19          1               40   \n",
         "2       4.812691     True          35          1               33   \n",
         "3       7.251004     True          30          2               38   \n",
         "4       6.206013    False          21          1               36   \n",
         "5       6.062712    False          33          1               40   \n",
         "6       7.500126    False          19          1               39   \n",
         "7       7.687519     True          23          1               41   \n",
         "8       8.875811     True          24          1               40   \n",
         "9       7.387690    False          28          1               38   \n",
         "\n",
         "           hash_values  \n",
         "0   563561248331884029  \n",
         "1  3487851893553562338  \n",
         "2  2669304657201106008  \n",
         "3  7076342771382320241  \n",
         "4  8828960867056723893  \n",
         "5  4280252324912833683  \n",
         "6  6090508671071281093  \n",
         "7  8708360030053768340  \n",
         "8  8530116731648975419  \n",
         "9  1776323475383399588  "
        ]
       },
       "execution_count": 8,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "data_query = \"\"\"\n",
      "SELECT\n",
      "    weight_pounds,\n",
      "    is_male,\n",
      "    mother_age,\n",
      "    plurality,\n",
      "    gestation_weeks,\n",
      "    FARM_FINGERPRINT(\n",
      "        CONCAT(\n",
      "            CAST(year AS STRING),\n",
      "            CAST(month AS STRING),\n",
      "            CAST(date AS STRING),\n",
      "            CAST(state AS STRING),\n",
      "            CAST(mother_birth_state AS STRING)\n",
      "        )\n",
      "    ) AS hash_values\n",
      "FROM\n",
      "    ({CTE_hash_cols_fixed})\n",
      "\"\"\".format(CTE_hash_cols_fixed=hash_cols_fixed_query)\n",
      "\n",
      "display_dataframe_head_from_query(data_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The next query is going to find the counts of each of the unique 657484 `hash_values`. This will be our first step at making actual hash buckets for our split via the `GROUP BY`."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 9,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>hash_values</th>\n",
         "      <th>num_records</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>6001926139587584124</td>\n",
         "      <td>19</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>6064126287360941757</td>\n",
         "      <td>758</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>6824828135709159935</td>\n",
         "      <td>72</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>3363240092080644183</td>\n",
         "      <td>631</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>2666158614438147859</td>\n",
         "      <td>964</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>2958542686973584093</td>\n",
         "      <td>363</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>8332670353336108110</td>\n",
         "      <td>47</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>1459116430691530322</td>\n",
         "      <td>52</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>8084544908979932787</td>\n",
         "      <td>7</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>2610866487448411172</td>\n",
         "      <td>23</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "           hash_values  num_records\n",
         "0  6001926139587584124           19\n",
         "1  6064126287360941757          758\n",
         "2  6824828135709159935           72\n",
         "3  3363240092080644183          631\n",
         "4  2666158614438147859          964\n",
         "5  2958542686973584093          363\n",
         "6  8332670353336108110           47\n",
         "7  1459116430691530322           52\n",
         "8  8084544908979932787            7\n",
         "9  2610866487448411172           23"
        ]
       },
       "execution_count": 9,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Get the counts of each of the unique hash of our splitting column\n",
      "first_bucketing_query = \"\"\"\n",
      "SELECT\n",
      "    hash_values,\n",
      "    COUNT(*) AS num_records\n",
      "FROM\n",
      "    ({CTE_data})\n",
      "GROUP BY\n",
      "    hash_values\n",
      "\"\"\".format(CTE_data=data_query)\n",
      "\n",
      "display_dataframe_head_from_query(first_bucketing_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The query below performs a second layer of bucketing where now for each of these bucket indices we count the number of records."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 10,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>17</td>\n",
         "      <td>222562</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>46</td>\n",
         "      <td>281627</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>7</td>\n",
         "      <td>270933</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>85</td>\n",
         "      <td>368045</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>40</td>\n",
         "      <td>333712</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>19</td>\n",
         "      <td>384793</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>77</td>\n",
         "      <td>401941</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>95</td>\n",
         "      <td>313544</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>81</td>\n",
         "      <td>233538</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>24</td>\n",
         "      <td>352559</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   bucket_index  num_records\n",
         "0            17       222562\n",
         "1            46       281627\n",
         "2             7       270933\n",
         "3            85       368045\n",
         "4            40       333712\n",
         "5            19       384793\n",
         "6            77       401941\n",
         "7            95       313544\n",
         "8            81       233538\n",
         "9            24       352559"
        ]
       },
       "execution_count": 10,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Get the number of records in each of the hash buckets\n",
      "second_bucketing_query = \"\"\"\n",
      "SELECT\n",
      "    ABS(MOD(hash_values, {modulo_divisor})) AS bucket_index,\n",
      "    SUM(num_records) AS num_records\n",
      "FROM\n",
      "    ({CTE_first_bucketing})\n",
      "GROUP BY\n",
      "    ABS(MOD(hash_values, {modulo_divisor}))\n",
      "\"\"\".format(\n",
      "    CTE_first_bucketing=first_bucketing_query, modulo_divisor=modulo_divisor)\n",
      "\n",
      "display_dataframe_head_from_query(second_bucketing_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The number of records is hard for us to easily understand the split, so we will normalize the count into percentage of the data in each of the hash buckets in the next query."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 11,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>4</td>\n",
         "      <td>398118</td>\n",
         "      <td>0.012060</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>92</td>\n",
         "      <td>336735</td>\n",
         "      <td>0.010201</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>70</td>\n",
         "      <td>285539</td>\n",
         "      <td>0.008650</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>78</td>\n",
         "      <td>326758</td>\n",
         "      <td>0.009898</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>16</td>\n",
         "      <td>172145</td>\n",
         "      <td>0.005215</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>94</td>\n",
         "      <td>431001</td>\n",
         "      <td>0.013056</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>5</td>\n",
         "      <td>449280</td>\n",
         "      <td>0.013610</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>62</td>\n",
         "      <td>426834</td>\n",
         "      <td>0.012930</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>30</td>\n",
         "      <td>333513</td>\n",
         "      <td>0.010103</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>34</td>\n",
         "      <td>379000</td>\n",
         "      <td>0.011481</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   bucket_index  num_records  percent_records\n",
         "0             4       398118         0.012060\n",
         "1            92       336735         0.010201\n",
         "2            70       285539         0.008650\n",
         "3            78       326758         0.009898\n",
         "4            16       172145         0.005215\n",
         "5            94       431001         0.013056\n",
         "6             5       449280         0.013610\n",
         "7            62       426834         0.012930\n",
         "8            30       333513         0.010103\n",
         "9            34       379000         0.011481"
        ]
       },
       "execution_count": 11,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Calculate the overall percentages\n",
      "percentages_query = \"\"\"\n",
      "SELECT\n",
      "    bucket_index,\n",
      "    num_records,\n",
      "    CAST(num_records AS FLOAT64) / (\n",
      "    SELECT\n",
      "        SUM(num_records)\n",
      "    FROM\n",
      "        ({CTE_second_bucketing})) AS percent_records\n",
      "FROM\n",
      "    ({CTE_second_bucketing})\n",
      "\"\"\".format(CTE_second_bucketing=second_bucketing_query)\n",
      "\n",
      "display_dataframe_head_from_query(percentages_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We'll now select the range of buckets to be used in training."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 12,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "      <th>dataset_name</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>52</td>\n",
         "      <td>204972</td>\n",
         "      <td>0.006209</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>33</td>\n",
         "      <td>410226</td>\n",
         "      <td>0.012427</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>23</td>\n",
         "      <td>559019</td>\n",
         "      <td>0.016934</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>28</td>\n",
         "      <td>449682</td>\n",
         "      <td>0.013622</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>62</td>\n",
         "      <td>426834</td>\n",
         "      <td>0.012930</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>73</td>\n",
         "      <td>411771</td>\n",
         "      <td>0.012474</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>38</td>\n",
         "      <td>338150</td>\n",
         "      <td>0.010243</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>35</td>\n",
         "      <td>250505</td>\n",
         "      <td>0.007588</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>65</td>\n",
         "      <td>289303</td>\n",
         "      <td>0.008764</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>61</td>\n",
         "      <td>453904</td>\n",
         "      <td>0.013750</td>\n",
         "      <td>train</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   bucket_index  num_records  percent_records dataset_name\n",
         "0            52       204972         0.006209        train\n",
         "1            33       410226         0.012427        train\n",
         "2            23       559019         0.016934        train\n",
         "3            28       449682         0.013622        train\n",
         "4            62       426834         0.012930        train\n",
         "5            73       411771         0.012474        train\n",
         "6            38       338150         0.010243        train\n",
         "7            35       250505         0.007588        train\n",
         "8            65       289303         0.008764        train\n",
         "9            61       453904         0.013750        train"
        ]
       },
       "execution_count": 12,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Choose hash buckets for training and pull in their statistics\n",
      "train_query = \"\"\"\n",
      "SELECT\n",
      "    *,\n",
      "    \"train\" AS dataset_name\n",
      "FROM\n",
      "    ({CTE_percentages})\n",
      "WHERE\n",
      "    bucket_index >= 0\n",
      "    AND bucket_index < {train_buckets}\n",
      "\"\"\".format(\n",
      "    CTE_percentages=percentages_query,\n",
      "    train_buckets=train_buckets)\n",
      "\n",
      "display_dataframe_head_from_query(train_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We'll do the same by selecting the range of buckets to be used evaluation."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 13,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "      <th>dataset_name</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>80</td>\n",
         "      <td>312489</td>\n",
         "      <td>0.009466</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>83</td>\n",
         "      <td>411258</td>\n",
         "      <td>0.012458</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>85</td>\n",
         "      <td>368045</td>\n",
         "      <td>0.011149</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>82</td>\n",
         "      <td>468179</td>\n",
         "      <td>0.014182</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>87</td>\n",
         "      <td>523881</td>\n",
         "      <td>0.015870</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>88</td>\n",
         "      <td>423809</td>\n",
         "      <td>0.012838</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>86</td>\n",
         "      <td>274489</td>\n",
         "      <td>0.008315</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>89</td>\n",
         "      <td>256482</td>\n",
         "      <td>0.007770</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>81</td>\n",
         "      <td>233538</td>\n",
         "      <td>0.007074</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>84</td>\n",
         "      <td>341155</td>\n",
         "      <td>0.010334</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   bucket_index  num_records  percent_records dataset_name\n",
         "0            80       312489         0.009466         eval\n",
         "1            83       411258         0.012458         eval\n",
         "2            85       368045         0.011149         eval\n",
         "3            82       468179         0.014182         eval\n",
         "4            87       523881         0.015870         eval\n",
         "5            88       423809         0.012838         eval\n",
         "6            86       274489         0.008315         eval\n",
         "7            89       256482         0.007770         eval\n",
         "8            81       233538         0.007074         eval\n",
         "9            84       341155         0.010334         eval"
        ]
       },
       "execution_count": 13,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Choose hash buckets for validation and pull in their statistics\n",
      "eval_query = \"\"\"\n",
      "SELECT\n",
      "    *,\n",
      "    \"eval\" AS dataset_name\n",
      "FROM\n",
      "    ({CTE_percentages})\n",
      "WHERE\n",
      "    bucket_index >= {train_buckets}\n",
      "    AND bucket_index < {cum_eval_buckets}\n",
      "\"\"\".format(\n",
      "    CTE_percentages=percentages_query,\n",
      "    train_buckets=train_buckets,\n",
      "    cum_eval_buckets=train_buckets + eval_buckets)\n",
      "\n",
      "display_dataframe_head_from_query(eval_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Lastly, we'll select the hash buckets to be used for the test split."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 14,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "      <th>dataset_name</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>92</td>\n",
         "      <td>336735</td>\n",
         "      <td>0.010201</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>91</td>\n",
         "      <td>333267</td>\n",
         "      <td>0.010096</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>90</td>\n",
         "      <td>286465</td>\n",
         "      <td>0.008678</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>94</td>\n",
         "      <td>431001</td>\n",
         "      <td>0.013056</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>93</td>\n",
         "      <td>215710</td>\n",
         "      <td>0.006534</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>99</td>\n",
         "      <td>223334</td>\n",
         "      <td>0.006765</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>95</td>\n",
         "      <td>313544</td>\n",
         "      <td>0.009498</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>97</td>\n",
         "      <td>480790</td>\n",
         "      <td>0.014564</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>96</td>\n",
         "      <td>529357</td>\n",
         "      <td>0.016036</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>98</td>\n",
         "      <td>374697</td>\n",
         "      <td>0.011351</td>\n",
         "      <td>test</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   bucket_index  num_records  percent_records dataset_name\n",
         "0            92       336735         0.010201         test\n",
         "1            91       333267         0.010096         test\n",
         "2            90       286465         0.008678         test\n",
         "3            94       431001         0.013056         test\n",
         "4            93       215710         0.006534         test\n",
         "5            99       223334         0.006765         test\n",
         "6            95       313544         0.009498         test\n",
         "7            97       480790         0.014564         test\n",
         "8            96       529357         0.016036         test\n",
         "9            98       374697         0.011351         test"
        ]
       },
       "execution_count": 14,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Choose hash buckets for testing and pull in their statistics\n",
      "test_query = \"\"\"\n",
      "SELECT\n",
      "    *,\n",
      "    \"test\" AS dataset_name\n",
      "FROM\n",
      "    ({CTE_percentages})\n",
      "WHERE\n",
      "    bucket_index >= {cum_eval_buckets}\n",
      "    AND bucket_index < {modulo_divisor}\n",
      "\"\"\".format(\n",
      "    CTE_percentages=percentages_query,\n",
      "    cum_eval_buckets=train_buckets + eval_buckets,\n",
      "    modulo_divisor=modulo_divisor)\n",
      "\n",
      "display_dataframe_head_from_query(test_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the below query, we'll `UNION ALL` all of the datasets together so that all three sets of hash buckets will be within one table. We added `dataset_id` so that we can sort on it in the query after."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 15,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>dataset_id</th>\n",
         "      <th>bucket_index</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "      <th>dataset_name</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>1</td>\n",
         "      <td>85</td>\n",
         "      <td>368045</td>\n",
         "      <td>0.011149</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>1</td>\n",
         "      <td>88</td>\n",
         "      <td>423809</td>\n",
         "      <td>0.012838</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>1</td>\n",
         "      <td>89</td>\n",
         "      <td>256482</td>\n",
         "      <td>0.007770</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>3</th>\n",
         "      <td>1</td>\n",
         "      <td>80</td>\n",
         "      <td>312489</td>\n",
         "      <td>0.009466</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>4</th>\n",
         "      <td>1</td>\n",
         "      <td>81</td>\n",
         "      <td>233538</td>\n",
         "      <td>0.007074</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>5</th>\n",
         "      <td>1</td>\n",
         "      <td>83</td>\n",
         "      <td>411258</td>\n",
         "      <td>0.012458</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>6</th>\n",
         "      <td>1</td>\n",
         "      <td>82</td>\n",
         "      <td>468179</td>\n",
         "      <td>0.014182</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>7</th>\n",
         "      <td>1</td>\n",
         "      <td>84</td>\n",
         "      <td>341155</td>\n",
         "      <td>0.010334</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>8</th>\n",
         "      <td>1</td>\n",
         "      <td>87</td>\n",
         "      <td>523881</td>\n",
         "      <td>0.015870</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>9</th>\n",
         "      <td>1</td>\n",
         "      <td>86</td>\n",
         "      <td>274489</td>\n",
         "      <td>0.008315</td>\n",
         "      <td>eval</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   dataset_id  bucket_index  num_records  percent_records dataset_name\n",
         "0           1            85       368045         0.011149         eval\n",
         "1           1            88       423809         0.012838         eval\n",
         "2           1            89       256482         0.007770         eval\n",
         "3           1            80       312489         0.009466         eval\n",
         "4           1            81       233538         0.007074         eval\n",
         "5           1            83       411258         0.012458         eval\n",
         "6           1            82       468179         0.014182         eval\n",
         "7           1            84       341155         0.010334         eval\n",
         "8           1            87       523881         0.015870         eval\n",
         "9           1            86       274489         0.008315         eval"
        ]
       },
       "execution_count": 15,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Union the training, validation, and testing dataset statistics\n",
      "union_query = \"\"\"\n",
      "SELECT\n",
      "    0 AS dataset_id,\n",
      "    *\n",
      "FROM\n",
      "    ({CTE_train})\n",
      "UNION ALL\n",
      "SELECT\n",
      "    1 AS dataset_id,\n",
      "    *\n",
      "FROM\n",
      "    ({CTE_eval})\n",
      "UNION ALL\n",
      "SELECT\n",
      "    2 AS dataset_id,\n",
      "    *\n",
      "FROM\n",
      "    ({CTE_test})\n",
      "\"\"\".format(CTE_train=train_query, CTE_eval=eval_query, CTE_test=test_query)\n",
      "\n",
      "display_dataframe_head_from_query(union_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Lastly, we'll show the final split between train, eval, and test sets. We can see both the number of records and percent of the total data. It is really close to that we were hoping to get."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 16,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>dataset_id</th>\n",
         "      <th>dataset_name</th>\n",
         "      <th>num_records</th>\n",
         "      <th>percent_records</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <th>0</th>\n",
         "      <td>0</td>\n",
         "      <td>train</td>\n",
         "      <td>25873134</td>\n",
         "      <td>0.783765</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>1</th>\n",
         "      <td>1</td>\n",
         "      <td>eval</td>\n",
         "      <td>3613325</td>\n",
         "      <td>0.109457</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <th>2</th>\n",
         "      <td>2</td>\n",
         "      <td>test</td>\n",
         "      <td>3524900</td>\n",
         "      <td>0.106778</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   dataset_id dataset_name  num_records  percent_records\n",
         "0           0        train     25873134         0.783765\n",
         "1           1         eval      3613325         0.109457\n",
         "2           2         test      3524900         0.106778"
        ]
       },
       "execution_count": 16,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "# Show final splitting and associated statistics\n",
      "split_query = \"\"\"\n",
      "SELECT\n",
      "    dataset_id,\n",
      "    dataset_name,\n",
      "    SUM(num_records) AS num_records,\n",
      "    SUM(percent_records) AS percent_records\n",
      "FROM\n",
      "    ({CTE_union})\n",
      "GROUP BY\n",
      "    dataset_id,\n",
      "    dataset_name\n",
      "ORDER BY\n",
      "    dataset_id\n",
      "\"\"\".format(CTE_union=union_query)\n",
      "\n",
      "display_dataframe_head_from_query(split_query)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Now that we know that our splitting values produce a good global splitting on our data, here's a way to get a well-distributed portion of the data in such a way that the train, eval, test sets do not overlap and takes a subsample of our global splits."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 33,
     "metadata": {},
     "outputs": [
      {
       "name": "stdout",
       "output_type": "stream",
       "text": [
        "There are 7733 examples in the train dataset.\n",
        "There are 1037 examples in the validation dataset.\n",
        "There are 561 examples in the test dataset.\n"
       ]
      }
     ],
     "source": [
      "# every_n allows us to subsample from each of the hash values\n",
      "# This helps us get approximately the record counts we want\n",
      "every_n = 1000\n",
      "\n",
      "splitting_string = \"ABS(MOD(hash_values, {0} * {1}))\".format(every_n, modulo_divisor)\n",
      "\n",
      "def create_data_split_sample_df(query_string, splitting_string, lo, up):\n",
      "    \"\"\"Creates a dataframe with a sample of a data split.\n",
      "\n",
      "    Args:\n",
      "        query_string: str, query to run to generate splits.\n",
      "        splitting_string: str, modulo string to split by.\n",
      "        lo: float, lower bound for bucket filtering for split.\n",
      "        up: float, upper bound for bucket filtering for split.\n",
      "    Returns:\n",
      "        Dataframe containing data split sample.\n",
      "    \"\"\"\n",
      "    query = \"SELECT * FROM ({0}) WHERE {1} >= {2} and {1} < {3}\".format(\n",
      "        query_string, splitting_string, int(lo), int(up))\n",
      "\n",
      "    df = bq.query(query).to_dataframe()\n",
      "\n",
      "    return df\n",
      "\n",
      "train_df = create_data_split_sample_df(\n",
      "    data_query, splitting_string,\n",
      "    lo=0, up=train_percent)\n",
      "\n",
      "eval_df = create_data_split_sample_df(\n",
      "    data_query, splitting_string,\n",
      "    lo=train_percent, up=train_percent + eval_percent)\n",
      "\n",
      "test_df = create_data_split_sample_df(\n",
      "    data_query, splitting_string,\n",
      "    lo=train_percent + eval_percent, up=modulo_divisor)\n",
      "\n",
      "print(\"There are {} examples in the train dataset.\".format(len(train_df)))\n",
      "print(\"There are {} examples in the validation dataset.\".format(len(eval_df)))\n",
      "print(\"There are {} examples in the test dataset.\".format(len(test_df)))"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Preprocess data using Pandas\n",
      "\n",
      "We'll perform a few preprocessing steps to the data in our dataset. Let's add extra rows to simulate the lack of ultrasound. That is we'll duplicate some rows and make the `is_male` field be `Unknown`. Also, if there is more than child we'll change the `plurality` to `Multiple(2+)`. While we're at it, we'll also change the plurality column to be a string. We'll perform these operations below. \n",
      "\n",
      "Let's start by examining the training dataset as is."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 34,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>is_male</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <td>0</td>\n",
         "      <td>9.499719</td>\n",
         "      <td>True</td>\n",
         "      <td>30</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>505732274561700014</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>1</td>\n",
         "      <td>6.027438</td>\n",
         "      <td>True</td>\n",
         "      <td>26</td>\n",
         "      <td>1</td>\n",
         "      <td>36</td>\n",
         "      <td>1409348435509100014</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>2</td>\n",
         "      <td>6.124442</td>\n",
         "      <td>True</td>\n",
         "      <td>34</td>\n",
         "      <td>2</td>\n",
         "      <td>37</td>\n",
         "      <td>2620860165093800008</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>3</td>\n",
         "      <td>9.001474</td>\n",
         "      <td>True</td>\n",
         "      <td>28</td>\n",
         "      <td>1</td>\n",
         "      <td>35</td>\n",
         "      <td>1409348435509100014</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>4</td>\n",
         "      <td>7.070225</td>\n",
         "      <td>False</td>\n",
         "      <td>23</td>\n",
         "      <td>1</td>\n",
         "      <td>40</td>\n",
         "      <td>4659354114038800077</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   weight_pounds  is_male  mother_age  plurality  gestation_weeks  \\\n",
         "0       9.499719     True          30          1               40   \n",
         "1       6.027438     True          26          1               36   \n",
         "2       6.124442     True          34          2               37   \n",
         "3       9.001474     True          28          1               35   \n",
         "4       7.070225    False          23          1               40   \n",
         "\n",
         "            hash_values  \n",
         "0   505732274561700014  \n",
         "1  1409348435509100014  \n",
         "2  2620860165093800008  \n",
         "3  1409348435509100014  \n",
         "4  4659354114038800077  "
        ]
       },
       "execution_count": 34,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "train_df.head()"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Also, notice that there are some very important numeric fields that are missing in some rows (the count in Pandas doesn't count missing data)"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 35,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <td>count</td>\n",
         "      <td>7733.000000</td>\n",
         "      <td>7733.000000</td>\n",
         "      <td>7733.000000</td>\n",
         "      <td>7733.000000</td>\n",
         "      <td>7.733000e+03</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>mean</td>\n",
         "      <td>7.264415</td>\n",
         "      <td>28.213371</td>\n",
         "      <td>1.035691</td>\n",
         "      <td>38.691064</td>\n",
         "      <td>4.983286e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>std</td>\n",
         "      <td>1.303220</td>\n",
         "      <td>6.134232</td>\n",
         "      <td>0.201568</td>\n",
         "      <td>2.531921</td>\n",
         "      <td>2.551244e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>min</td>\n",
         "      <td>0.562179</td>\n",
         "      <td>13.000000</td>\n",
         "      <td>1.000000</td>\n",
         "      <td>18.000000</td>\n",
         "      <td>5.826385e+15</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>25%</td>\n",
         "      <td>6.624891</td>\n",
         "      <td>23.000000</td>\n",
         "      <td>1.000000</td>\n",
         "      <td>38.000000</td>\n",
         "      <td>3.153609e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>50%</td>\n",
         "      <td>7.345803</td>\n",
         "      <td>28.000000</td>\n",
         "      <td>1.000000</td>\n",
         "      <td>39.000000</td>\n",
         "      <td>4.896699e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>75%</td>\n",
         "      <td>8.062305</td>\n",
         "      <td>33.000000</td>\n",
         "      <td>1.000000</td>\n",
         "      <td>40.000000</td>\n",
         "      <td>6.784884e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>max</td>\n",
         "      <td>11.563246</td>\n",
         "      <td>48.000000</td>\n",
         "      <td>4.000000</td>\n",
         "      <td>47.000000</td>\n",
         "      <td>9.210618e+18</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "       weight_pounds   mother_age    plurality  gestation_weeks    hash_values\n",
         "count    7733.000000  7733.000000  7733.000000      7733.000000  7.733000e+03\n",
         "mean        7.264415    28.213371     1.035691        38.691064  4.983286e+18\n",
         "std         1.303220     6.134232     0.201568         2.531921  2.551244e+18\n",
         "min         0.562179    13.000000     1.000000        18.000000  5.826385e+15\n",
         "25%         6.624891    23.000000     1.000000        38.000000  3.153609e+18\n",
         "50%         7.345803    28.000000     1.000000        39.000000  4.896699e+18\n",
         "75%         8.062305    33.000000     1.000000        40.000000  6.784884e+18\n",
         "max        11.563246    48.000000     4.000000        47.000000  9.210618e+18"
        ]
       },
       "execution_count": 35,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "train_df.describe()"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It is always crucial to clean raw data before using in machine learning, so we have a preprocessing step. We'll define a `preprocess` function below. Note that the mother's age is an input to our model so users will have to provide the mother's age; otherwise, our service won't work. The features we use for our model were chosen because they are such good predictors and because they are easy enough to collect."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 36,
     "metadata": {},
     "outputs": [],
     "source": [
      "def preprocess(df):\n",
      "    \"\"\" Preprocess pandas dataframe for augmented babyweight data.\n",
      "    \n",
      "    Args:\n",
      "        df: Dataframe containing raw babyweight data.\n",
      "    Returns:\n",
      "        Pandas dataframe containing preprocessed raw babyweight data as well\n",
      "            as simulated no ultrasound data masking some of the original data.\n",
      "    \"\"\"\n",
      "    # Clean up raw data\n",
      "    # Filter out what we don\"t want to use for training\n",
      "    df = df[df.weight_pounds > 0]\n",
      "    df = df[df.mother_age > 0]\n",
      "    df = df[df.gestation_weeks > 0]\n",
      "    df = df[df.plurality > 0]\n",
      "\n",
      "    # Modify plurality field to be a string\n",
      "    twins_etc = dict(zip([1,2,3,4,5],\n",
      "                   [\"Single(1)\",\n",
      "                    \"Twins(2)\",\n",
      "                    \"Triplets(3)\",\n",
      "                    \"Quadruplets(4)\",\n",
      "                    \"Quintuplets(5)\"]))\n",
      "    df[\"plurality\"].replace(twins_etc, inplace=True)\n",
      "\n",
      "    # Clone data and mask certain columns to simulate lack of ultrasound\n",
      "    no_ultrasound = df.copy(deep=True)\n",
      "\n",
      "    # Modify is_male\n",
      "    no_ultrasound[\"is_male\"] = \"Unknown\"\n",
      "    \n",
      "    # Modify plurality\n",
      "    condition = no_ultrasound[\"plurality\"] != \"Single(1)\"\n",
      "    no_ultrasound.loc[condition, \"plurality\"] = \"Multiple(2+)\"\n",
      "\n",
      "    # Concatenate both datasets together and shuffle\n",
      "    return pd.concat(\n",
      "        [df, no_ultrasound]).sample(frac=1).reset_index(drop=True)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's process the train, eval, test set and see a small sample of the training data after our preprocessing:"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 37,
     "metadata": {},
     "outputs": [],
     "source": [
      "train_df = preprocess(train_df)\n",
      "eval_df = preprocess(eval_df)\n",
      "test_df = preprocess(test_df)"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 38,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>is_male</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <td>0</td>\n",
         "      <td>7.874912</td>\n",
         "      <td>Unknown</td>\n",
         "      <td>38</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>38</td>\n",
         "      <td>8717259940738900003</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>1</td>\n",
         "      <td>8.999270</td>\n",
         "      <td>Unknown</td>\n",
         "      <td>31</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>45</td>\n",
         "      <td>6781866293108400060</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>2</td>\n",
         "      <td>7.251004</td>\n",
         "      <td>True</td>\n",
         "      <td>24</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>40</td>\n",
         "      <td>1696737464106800060</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>3</td>\n",
         "      <td>8.562754</td>\n",
         "      <td>True</td>\n",
         "      <td>43</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>39</td>\n",
         "      <td>4614303140002600076</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>4</td>\n",
         "      <td>6.194990</td>\n",
         "      <td>True</td>\n",
         "      <td>23</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>41</td>\n",
         "      <td>780565305641800050</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "   weight_pounds  is_male  mother_age  plurality  gestation_weeks  \\\n",
         "0       7.874912  Unknown          38  Single(1)               38   \n",
         "1       8.999270  Unknown          31  Single(1)               45   \n",
         "2       7.251004     True          24  Single(1)               40   \n",
         "3       8.562754     True          43  Single(1)               39   \n",
         "4       6.194990     True          23  Single(1)               41   \n",
         "\n",
         "            hash_values  \n",
         "0  8717259940738900003  \n",
         "1  6781866293108400060  \n",
         "2  1696737464106800060  \n",
         "3  4614303140002600076  \n",
         "4   780565305641800050  "
        ]
       },
       "execution_count": 38,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "train_df.head()"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 39,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>is_male</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>plurality</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <td>15461</td>\n",
         "      <td>7.251004</td>\n",
         "      <td>True</td>\n",
         "      <td>32</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>39</td>\n",
         "      <td>8655151740159000017</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>15462</td>\n",
         "      <td>8.811877</td>\n",
         "      <td>True</td>\n",
         "      <td>30</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>39</td>\n",
         "      <td>845203792559000058</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>15463</td>\n",
         "      <td>7.248799</td>\n",
         "      <td>True</td>\n",
         "      <td>26</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>40</td>\n",
         "      <td>1409348435509100014</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>15464</td>\n",
         "      <td>7.625790</td>\n",
         "      <td>Unknown</td>\n",
         "      <td>22</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>40</td>\n",
         "      <td>2875790318525700041</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>15465</td>\n",
         "      <td>6.499227</td>\n",
         "      <td>Unknown</td>\n",
         "      <td>22</td>\n",
         "      <td>Single(1)</td>\n",
         "      <td>38</td>\n",
         "      <td>8720767384765100051</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "       weight_pounds  is_male  mother_age  plurality  gestation_weeks  \\\n",
         "15461       7.251004     True          32  Single(1)               39   \n",
         "15462       8.811877     True          30  Single(1)               39   \n",
         "15463       7.248799     True          26  Single(1)               40   \n",
         "15464       7.625790  Unknown          22  Single(1)               40   \n",
         "15465       6.499227  Unknown          22  Single(1)               38   \n",
         "\n",
         "                hash_values  \n",
         "15461  8655151740159000017  \n",
         "15462   845203792559000058  \n",
         "15463  1409348435509100014  \n",
         "15464  2875790318525700041  \n",
         "15465  8720767384765100051  "
        ]
       },
       "execution_count": 39,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "train_df.tail()"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Let's look again at a summary of the dataset. Note that we only see numeric columns, so `plurality` does not show up."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 40,
     "metadata": {},
     "outputs": [
      {
       "data": {
        "text/html": [
         "<div>\n",
         "<style scoped>\n",
         "    .dataframe tbody tr th:only-of-type {\n",
         "        vertical-align: middle;\n",
         "    }\n",
         "\n",
         "    .dataframe tbody tr th {\n",
         "        vertical-align: top;\n",
         "    }\n",
         "\n",
         "    .dataframe thead th {\n",
         "        text-align: right;\n",
         "    }\n",
         "</style>\n",
         "<table border=\"1\" class=\"dataframe\">\n",
         "  <thead>\n",
         "    <tr style=\"text-align: right;\">\n",
         "      <th></th>\n",
         "      <th>weight_pounds</th>\n",
         "      <th>mother_age</th>\n",
         "      <th>gestation_weeks</th>\n",
         "      <th>hash_values</th>\n",
         "    </tr>\n",
         "  </thead>\n",
         "  <tbody>\n",
         "    <tr>\n",
         "      <td>count</td>\n",
         "      <td>15466.000000</td>\n",
         "      <td>15466.000000</td>\n",
         "      <td>15466.000000</td>\n",
         "      <td>1.546600e+04</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>mean</td>\n",
         "      <td>7.264415</td>\n",
         "      <td>28.213371</td>\n",
         "      <td>38.691064</td>\n",
         "      <td>4.983286e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>std</td>\n",
         "      <td>1.303178</td>\n",
         "      <td>6.134034</td>\n",
         "      <td>2.531839</td>\n",
         "      <td>2.551162e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>min</td>\n",
         "      <td>0.562179</td>\n",
         "      <td>13.000000</td>\n",
         "      <td>18.000000</td>\n",
         "      <td>5.826385e+15</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>25%</td>\n",
         "      <td>6.624891</td>\n",
         "      <td>23.000000</td>\n",
         "      <td>38.000000</td>\n",
         "      <td>3.153609e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>50%</td>\n",
         "      <td>7.345803</td>\n",
         "      <td>28.000000</td>\n",
         "      <td>39.000000</td>\n",
         "      <td>4.896699e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>75%</td>\n",
         "      <td>8.062305</td>\n",
         "      <td>33.000000</td>\n",
         "      <td>40.000000</td>\n",
         "      <td>6.784884e+18</td>\n",
         "    </tr>\n",
         "    <tr>\n",
         "      <td>max</td>\n",
         "      <td>11.563246</td>\n",
         "      <td>48.000000</td>\n",
         "      <td>47.000000</td>\n",
         "      <td>9.210618e+18</td>\n",
         "    </tr>\n",
         "  </tbody>\n",
         "</table>\n",
         "</div>"
        ],
        "text/plain": [
         "       weight_pounds    mother_age  gestation_weeks    hash_values\n",
         "count   15466.000000  15466.000000     15466.000000  1.546600e+04\n",
         "mean        7.264415     28.213371        38.691064  4.983286e+18\n",
         "std         1.303178      6.134034         2.531839  2.551162e+18\n",
         "min         0.562179     13.000000        18.000000  5.826385e+15\n",
         "25%         6.624891     23.000000        38.000000  3.153609e+18\n",
         "50%         7.345803     28.000000        39.000000  4.896699e+18\n",
         "75%         8.062305     33.000000        40.000000  6.784884e+18\n",
         "max        11.563246     48.000000        47.000000  9.210618e+18"
        ]
       },
       "execution_count": 40,
       "metadata": {},
       "output_type": "execute_result"
      }
     ],
     "source": [
      "train_df.describe()"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Write to .csv files \n",
      "\n",
      "In the final versions, we want to read from files, not Pandas dataframes. So, we write the Pandas dataframes out as csv files. Using csv files gives us the advantage of shuffling during read. This is important for distributed training because some workers might be slower than others, and shuffling the data helps prevent the same data from being assigned to the slow workers."
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 41,
     "metadata": {},
     "outputs": [],
     "source": [
      "# Define columns\n",
      "columns = [\"weight_pounds\",\n",
      "           \"is_male\",\n",
      "           \"mother_age\",\n",
      "           \"plurality\",\n",
      "           \"gestation_weeks\"]\n",
      "\n",
      "# Write out CSV files\n",
      "train_df.to_csv(\n",
      "    path_or_buf=\"train.csv\", columns=columns, header=False, index=False)\n",
      "eval_df.to_csv(\n",
      "    path_or_buf=\"eval.csv\", columns=columns, header=False, index=False)\n",
      "test_df.to_csv(\n",
      "    path_or_buf=\"test.csv\", columns=columns, header=False, index=False)"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 42,
     "metadata": {},
     "outputs": [
      {
       "name": "stdout",
       "output_type": "stream",
       "text": [
        "  2074 eval.csv\n",
        "  1122 test.csv\n",
        " 15466 train.csv\n",
        " 18662 total\n"
       ]
      }
     ],
     "source": [
      "%%bash\n",
      "wc -l *.csv"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 43,
     "metadata": {},
     "outputs": [
      {
       "name": "stdout",
       "output_type": "stream",
       "text": [
        "==> eval.csv <==\n",
        "8.62448368944,Unknown,31,Single(1),42\n",
        "6.9996768185,Unknown,32,Single(1),39\n",
        "6.6248909731,False,30,Single(1),38\n",
        "8.3114272774,False,19,Single(1),41\n",
        "8.313631900019999,True,32,Single(1),37\n",
        "7.06140625186,Unknown,34,Single(1),41\n",
        "7.62578964258,Unknown,34,Single(1),39\n",
        "7.3744626639,Unknown,20,Single(1),39\n",
        "1.93786328298,False,32,Triplets(3),28\n",
        "8.99926953484,True,34,Single(1),39\n",
        "\n",
        "==> test.csv <==\n",
        "7.3744626639,Unknown,25,Single(1),44\n",
        "6.93794738514,Unknown,24,Single(1),40\n",
        "6.87621795178,True,30,Single(1),39\n",
        "6.87621795178,Unknown,29,Single(1),39\n",
        "7.0327461578,Unknown,36,Single(1),38\n",
        "9.31232594688,False,25,Single(1),39\n",
        "7.936641432,True,23,Single(1),37\n",
        "4.7840310854,Unknown,34,Multiple(2+),38\n",
        "7.31273323054,True,23,Single(1),39\n",
        "8.24969784404,False,32,Single(1),39\n",
        "\n",
        "==> train.csv <==\n",
        "7.87491199864,Unknown,38,Single(1),38\n",
        "8.99926953484,Unknown,31,Single(1),45\n",
        "7.25100379718,True,24,Single(1),40\n",
        "8.56275425608,True,43,Single(1),39\n",
        "6.1949895622,True,23,Single(1),41\n",
        "9.0609989682,Unknown,24,Single(1),38\n",
        "7.5618555866,True,26,Single(1),41\n",
        "7.30611936268,False,31,Single(1),41\n",
        "9.6672701887,True,29,Single(1),40\n",
        "6.4992274837599995,True,22,Single(1),39\n"
       ]
      }
     ],
     "source": [
      "%%bash\n",
      "head *.csv"
     ]
    },
    {
     "cell_type": "code",
     "execution_count": 44,
     "metadata": {},
     "outputs": [
      {
       "name": "stdout",
       "output_type": "stream",
       "text": [
        "==> eval.csv <==\n",
        "7.43839671988,False,25,Single(1),37\n",
        "7.06140625186,True,34,Single(1),41\n",
        "7.43619209726,True,36,Single(1),40\n",
        "3.56267015392,True,35,Twins(2),31\n",
        "8.811876612139999,False,27,Single(1),36\n",
        "8.0689187892,Unknown,36,Single(1),40\n",
        "8.7633749145,Unknown,34,Single(1),39\n",
        "7.43839671988,True,43,Single(1),40\n",
        "4.62529825676,Unknown,38,Multiple(2+),35\n",
        "6.1839664491,Unknown,20,Single(1),38\n",
        "\n",
        "==> test.csv <==\n",
        "6.37576861704,Unknown,21,Single(1),39\n",
        "7.5618555866,True,22,Single(1),39\n",
        "8.99926953484,Unknown,28,Single(1),42\n",
        "7.82420567838,Unknown,24,Single(1),39\n",
        "9.25059651352,True,26,Single(1),40\n",
        "8.62448368944,Unknown,28,Single(1),39\n",
        "5.2580249487,False,18,Single(1),38\n",
        "7.87491199864,True,25,Single(1),37\n",
        "5.81138522632,Unknown,41,Single(1),36\n",
        "6.93794738514,True,24,Single(1),40\n",
        "\n",
        "==> train.csv <==\n",
        "7.81318256528,True,18,Single(1),43\n",
        "7.31273323054,False,35,Single(1),34\n",
        "6.75055446244,Unknown,37,Single(1),39\n",
        "7.43839671988,True,32,Single(1),39\n",
        "6.9666074791999995,True,20,Single(1),38\n",
        "7.25100379718,True,32,Single(1),39\n",
        "8.811876612139999,True,30,Single(1),39\n",
        "7.24879917456,True,26,Single(1),40\n",
        "7.62578964258,Unknown,22,Single(1),40\n",
        "6.4992274837599995,Unknown,22,Single(1),38\n"
       ]
      }
     ],
     "source": [
      "%%bash\n",
      "tail *.csv"
     ]
    },
    {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eval.csv\n",
      "test.csv\n",
      "train.csv\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "ls *.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==> eval.csv <==\n",
      "6.87621795178,False,33,Single(1),40\n",
      "7.7492485093,Unknown,21,Single(1),38\n",
      "8.86699217764,False,22,Single(1),38\n",
      "6.60504936952,False,32,Single(1),40\n",
      "8.313631900019999,True,36,Single(1),39\n",
      "\n",
      "==> test.csv <==\n",
      "7.5618555866,True,40,Twins(2),43\n",
      "9.3586230219,Unknown,22,Single(1),40\n",
      "8.5539357656,True,26,Single(1),37\n",
      "5.81138522632,Unknown,36,Multiple(2+),36\n",
      "7.06140625186,Unknown,23,Single(1),40\n",
      "\n",
      "==> train.csv <==\n",
      "10.18756112702,Unknown,23,Single(1),33\n",
      "8.93754010148,True,40,Single(1),41\n",
      "6.9996768185,Unknown,23,Single(1),38\n",
      "8.65975765136,Unknown,19,Single(1),42\n",
      "4.2549216566,True,20,Single(1),33\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "head -5 *.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Keras model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set CSV Columns, label column, and column defaults.\n",
    "\n",
    "Now that we have verified that our CSV files exist, we need to set a few things that we will be using in our input function.\n",
    "* `CSV_COLUMNS` is going to be our header name of our column. Make sure that they are in the same order as in the CSV files\n",
    "* `LABEL_COLUMN` is the header name of the column that is our label. We will need to know this to pop it from our features dictionary.\n",
    "* `DEFAULTS` is a list with the same length as `CSV_COLUMNS`, i.e. there is a default for each column in our CSVs. Each element is a list itself with the default value for that CSV column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Determine CSV, label, and key columns\n",
    "# Create list of string column headers, make sure order matches.\n",
    "CSV_COLUMNS = [\"weight_pounds\",\n",
    "               \"is_male\",\n",
    "               \"mother_age\",\n",
    "               \"plurality\",\n",
    "               \"gestation_weeks\"]\n",
    "\n",
    "# Add string name for label column\n",
    "LABEL_COLUMN = \"weight_pounds\"\n",
    "\n",
    "# Set default values for each CSV column as a list of lists.\n",
    "# Treat is_male and plurality as strings.\n",
    "DEFAULTS = [[0.0], [\"null\"], [0.0], [\"null\"], [0.0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make dataset of features and label from CSV files.\n",
    "\n",
    "Next, we will write an input_fn to read the data. Since we are reading from CSV files we can save ourselves from trying to recreate the wheel and can use `tf.data.experimental.make_csv_dataset`. This will create a CSV dataset object. However we will need to divide the columns up into features and a label. We can do this by applying the map method to our dataset and popping our label column off of our dictionary of feature tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def features_and_labels(row_data):\n",
    "    \"\"\"Splits features and labels from feature dictionary.\n",
    "\n",
    "    Args:\n",
    "        row_data: Dictionary of CSV column names and tensor values.\n",
    "    Returns:\n",
    "        Dictionary of feature tensors and label tensor.\n",
    "    \"\"\"\n",
    "    label = row_data.pop(LABEL_COLUMN)\n",
    "\n",
    "    return row_data, label  # features, label\n",
    "\n",
    "\n",
    "def load_dataset(pattern, batch_size=1, mode='eval'):\n",
    "    \"\"\"Loads dataset using the tf.data API from CSV files.\n",
    "\n",
    "    Args:\n",
    "        pattern: str, file pattern to glob into list of files.\n",
    "        batch_size: int, the number of examples per batch.\n",
    "        mode: 'train' | 'eval' to determine if training or evaluating.\n",
    "    Returns:\n",
    "        `Dataset` object.\n",
    "    \"\"\"\n",
    "    # Make a CSV dataset\n",
    "    dataset = tf.data.experimental.make_csv_dataset(\n",
    "        file_pattern=pattern,\n",
    "        batch_size=batch_size,\n",
    "        column_names=CSV_COLUMNS,\n",
    "        column_defaults=DEFAULTS,\n",
    "        ignore_errors=True)\n",
    "\n",
    "    # Map dataset to features and label\n",
    "    dataset = dataset.map(map_func=features_and_labels)  # features, label\n",
    "\n",
    "    # Shuffle and repeat for training\n",
    "    if mode == 'train':\n",
    "        dataset = dataset.shuffle(buffer_size=1000).repeat()\n",
    "\n",
    "    # Take advantage of multi-threading; 1=AUTOTUNE\n",
    "    dataset = dataset.prefetch(buffer_size=1)\n",
    "\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create input layers for raw features.\n",
    "\n",
    "We'll need to get the data to read in by our input function to our model function, but just how do we go about connecting the dots? We can use Keras input layers [(tf.Keras.layers.Input)](https://www.tensorflow.org/api_docs/python/tf/keras/Input) by defining:\n",
    "* shape: A shape tuple (integers), not including the batch size. For instance, shape=(32,) indicates that the expected input will be batches of 32-dimensional vectors. Elements of this tuple can be None; 'None' elements represent dimensions where the shape is not known.\n",
    "* name: An optional name string for the layer. Should be unique in a model (do not reuse the same name twice). It will be autogenerated if it isn't provided.\n",
    "* dtype: The data type expected by the input, as a string (float32, float64, int32...)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "**Lab Task #1:** Creating input layers for raw features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 1\n",
    "# TODO -- Your code here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create feature columns for inputs.\n",
    "\n",
    "Next, define the feature columns. `mother_age` and `gestation_weeks` should be numeric. The others, `is_male` and `plurality`, should be categorical. Remember, only dense feature columns can be inputs to a DNN."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "**Lab Task #2:** Creating feature columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 2\n",
    "# TODO -- Your code here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create DNN dense hidden layers and output layer.\n",
    "\n",
    "So we've figured out how to get our inputs ready for machine learning but now we need to connect them to our desired output. Our model architecture is what links the two together. Let's create some hidden dense layers beginning with our inputs and end with a dense output layer. This is regression so make sure the output layer activation is correct and that the shape is right."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "**Lab Task #3:** Creating DNN dense hidden layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 3\n",
    "def get_model_outputs(inputs):\n",
    "    \"\"\"Creates model architecture and returns outputs.\n",
    "\n",
    "    Args:\n",
    "        inputs: Dense tensor used as inputs to model.\n",
    "    Returns:\n",
    "        Dense tensor output from the model.\n",
    "    \"\"\"\n",
    "    # Create two hidden layers of [64, 32] just in like the BQML DNN\n",
    "    # TODO -- Your code here.\n",
    "\n",
    "    # Final output is a linear activation because this is regression\n",
    "    output = tf.keras.layers.Dense(\n",
    "        units=1, activation=\"linear\", name=\"weight\")(h2)\n",
    "\n",
    "    return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create custom evaluation metric.\n",
    "\n",
    "We want to make sure that we have some useful way to measure model performance for us. Since this is regression, we would like to know the RMSE of the model on our evaluation dataset, however, this does not exist as a standard evaluation metric, so we'll have to create our own by using the true and predicted labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rmse(y_true, y_pred):\n",
    "    \"\"\"Calculates RMSE evaluation metric.\n",
    "\n",
    "    Args:\n",
    "        y_true: tensor, true labels.\n",
    "        y_pred: tensor, predicted labels.\n",
    "    Returns:\n",
    "        Tensor with value of RMSE between true and predicted labels.\n",
    "    \"\"\"\n",
    "    return tf.sqrt(tf.reduce_mean((y_pred - y_true) ** 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build DNN model tying all of the pieces together.\n",
    "\n",
    "Excellent! We've assembled all of the pieces, now we just need to tie them all together into a Keras Model. This is a simple feedforward model with no branching, side inputs, etc. so we could have used Keras' Sequential Model API but just for fun we're going to use Keras' Functional Model API. Here we will build the model using [tf.keras.models.Model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) giving our inputs and outputs and then compile our model with an optimizer, a loss function, and evaluation metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "**Lab Task #4:** Building DNN model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here is our DNN architecture so far:\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4276: IndicatorColumn._variable_shape (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow_core/python/feature_column/feature_column_v2.py:4331: VocabularyListCategoricalColumn._num_buckets (from tensorflow.python.feature_column.feature_column_v2) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "The old _FeatureColumn APIs are being deprecated. Please use the new FeatureColumn APIs instead.\n",
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "gestation_weeks (InputLayer)    [(None,)]            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "is_male (InputLayer)            [(None,)]            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "mother_age (InputLayer)         [(None,)]            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "plurality (InputLayer)          [(None,)]            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "dense_features (DenseFeatures)  (None, 11)           0           gestation_weeks[0][0]            \n",
      "                                                                 is_male[0][0]                    \n",
      "                                                                 mother_age[0][0]                 \n",
      "                                                                 plurality[0][0]                  \n",
      "__________________________________________________________________________________________________\n",
      "h1 (Dense)                      (None, 64)           768         dense_features[0][0]             \n",
      "__________________________________________________________________________________________________\n",
      "h2 (Dense)                      (None, 32)           2080        h1[0][0]                         \n",
      "__________________________________________________________________________________________________\n",
      "weight (Dense)                  (None, 1)            33          h2[0][0]                         \n",
      "==================================================================================================\n",
      "Total params: 2,881\n",
      "Trainable params: 2,881\n",
      "Non-trainable params: 0\n",
      "__________________________________________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# TODO 4\n",
    "# TODO -- Your code here.\n",
    "\n",
    "print(\"Here is our DNN architecture so far:\\n\")\n",
    "model = build_dnn_model()\n",
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize the DNN using the Keras plot_model utility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+MAAAEYCAYAAAA3TicjAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVhUZf8/8Pewb6KgoiLIooiJbaaYCxolXZVgPSmQ5pKiuGaKmeSWZVFYGm5l2uKeQvUTtcy1TAkft0xNQwtBwBVkEVCW4fP7w++cR2QbEGYGeL+uay7lzFnec5975vDhzLmPSkQERERERERERKQrMUb6TkBERERERETU2LAYJyIiIiIiItIxFuNEREREREREOsZinIiIiIiIiEjHTO6fkJSUhLfffhtqtVofeYioEoGBgQgMDNR3DCIiIiIiekBlzowfOXIEmzdv1kcWIqpEfHw8YmJi9B2DiIiIiIhqQZkz4xrR0dG6zEFEVQgKCtJ3BCIiIiIiqiW8ZpyIiIiIiIhIx1iMExEREREREekYi3EiIiIiIiIiHWMxTkRERERERKRjLMaJiIiIiIiIdIzFOBEREREREZGOsRgnIiIiIiIi0jEW40REREREREQ6xmKciIiIiIiISMdYjBMRERERERHpGItxIiIiIiIiIh1jMU5ERERERESkYyzGiYiIiIiIiHSMxTgRERERERGRjrEYb+AKCwtx6NAhfccgIiIiIiKiezS6YrxHjx6YMWOGvmPUuZs3b+Ltt9+GnZ0dfHx86nRbBw8eRHh4OFQqFVQqFUaMGIHY2Ng63aY2fvnlFwQGBiq5xo0bh7i4OH3HIiIiIiIigkpE5N4J0dHRCA4Oxn2T9S4lJQXOzs4PvMwrr7wCDw8PLFiwoDbjGSQRQatWrXDjxg2d7E8XFxdcunQJeXl5sLKyqvPtlef+fZ6fnw9ra2u0a9cOycnJeslUW4KCggDcfY8SEREREVG9FlMvzoxfvHgRQ4cOrZVlNm/e3CgKcQBQqVSwt7fX2fYsLS0BQG+FeHn7XJNFk42IiIiIiMgQmOg7QFVSU1Ph7+8PtVpdp8tQ/cZ9TkRERERE9UmtnBkXESxbtgzDhg3DhAkTYG5urlynq1KpAAC3b99GZGQkQkJC0K1bN/Tv3x+nT59W1nH06FH06NEDkyZNwty5c2FiYoLc3FysWbMGZ8+exdWrVzF+/Hhl/vPnz2Pw4MGYOXMmhg8fDh8fH5w6dQoAyl1GrVYjOjoaI0eORN++fZX1ZGdn46233kJ4eDjCwsLw7LPPIiwsDJmZmRARxMbGIjQ0FE5OTsjMzMTIkSPRvHlzdOnSBceOHdO6jfbv3w8zMzPY2Njgt99+Q1ZWFoYNGwaVSoWnnnoKZ86cAQCcOHECbdq0wRdffKFVu1X1/P0++eQTmJubY/r06crAbhW1vSa3k5MTDhw4oPVrBaBV24kI4uPjMX36dLi6uuLq1asYNGgQ7O3t0aVLF3z//fcAgFWrVpXqSzk5OVi0aFGpaRX1k+qorE9t2LABVlZWUKlU+Oijj1BcXAwA2LhxI8zMzLBmzRoAFe8PtVqNX3/9FVOnToWrqyvS0tLQr18/tGvXDpmZmTXKS0RERERE9ZjcZ8uWLVLO5EotXbpUjIyMJD09XUREIiIiBICEhYUp84wZM0bOnTun/Ozn5ycODg6SnZ0tIiIeHh5iZ2cnJSUlIiISFBQk165dk/+7pl08PT1LbbNDhw7i7u4uIiKFhYXStGlT8fLyUp4vb5nk5ORS03NycsTDw0PeeecdZZ5r166Jh4eHuLm5yc2bNyUlJUWsra0FgLz//vuSlJQk69evFwDi7e1drXaaMGGCmJubS1ZWloiI5Ofni4ODg7z66qvKPEVFReLj46O0Q1XtVtXznp6eyv7MyMiQYcOGyZ9//lkqV2Vtv3XrVrG0tJRt27ZV+fru3VZJSUmVbVdcXCzbt28XCwsLASCTJ0+WAwcOyMaNG8XGxkYAyKFDh0RExN3dvUy/vH9aefu8sun3q6pPzZ49WwDImTNnlGnJycny0ksvKT9XtD+uX78ucXFxYmlpKQAkIiJC9uzZIyEhIXLr1q0qs4mIBAYGSmBgoFbzEhERERGRQYuulWI8ICBAVCqVFBQUiIjI6dOnBYD06NFDREQOHz4sAMp9bN++XUREWrRoIQAkKipK1Gq1nD59WikoyyumFi1aJJs2bRIREbVaLe7u7mJiYqI8X94yJSUlpabPmjVLAMjly5dLzbd27VoBIDNmzBARkY4dO5Zqk5KSEnFwcBAzM7NqtdNff/0lAGTFihWl2s7a2lpycnJERCQ2NlZWrlypVbtp066aAvnff/+V0aNHy/Xr18vkqqztRe7+gUAb9xbjGtq0nYeHhwCQ3NxcZdqnn34qACQ4OLjCdd8/7UGL8ar6VHp6utjY2MiYMWOUaREREUpba7M/NO2RkZFRZZ77sRgnIiIiImowomvla+p+fn4QEfz4448AAAsLCwDA008/DeDu16C9vLwgImUe/v7+AIDPP/8cNjY2mDp1Kry9vZGbmwtbW9sKtxkWFoaAgACsWLECH3zwAQoKCpSvDldE85VmDc1trpo0aVJquuZr7L///nu5y6lUKtjZ2aGwsLDS7d2vc+fO8PX1xRdffAERQVJSEtRqNQoLC/Htt98CANatW4dhw4YBqLrdtGlXjQEDBiAvLw8tWrQok6uqtjcxqfnQAtq0nZHR3W5obW2tTBs4cCAA4MKFCzXednVV1aeaN2+O119/HWvXrkVaWhpEBPv27cNzzz0HQLt+rmkPXQ6sR0REREREhqdWivHJkydj9erVCAkJwZtvvonp06fj3XffxXvvvQcAyMjIQGJiIvLy8sosqxlwa/DgwTh58iSeffZZHD9+HD4+Psp1uOU5cuQIHn74Ybi7u2Pu3LmwsbGpdm5NEZiUlFRqeqtWrQAATZs2rfY6qzJ58mScOnUKR48eRWRkJBYuXIiXX34Zq1evxl9//QVXV1elKK2q3bRpV41PPvkEW7ZsQWRkZJl5q9v2uuDo6AgA1b6dXU1cv34dRUVFWvWpsLAwmJmZISoqCsePH4e3t7fyx4rq7A8iIiIiImrcaqUYV6vVOHPmDA4fPoxPPvkEsbGxmDdvnlKkdOrUSRnY6l5nz57F8uXLAQDz5s1D+/btsWvXLmzatAnFxcWYM2eOMu/9Z71HjBiBoqIiPP/88wCAkpISACh1P+2qzpRrzoBrzuhrpKSkAAD69++vXQNUw8CBA+Hk5IT58+cjLy8PXl5eGD9+PI4dO4aJEydiwoQJyrxVtZs27aoxYMAAzJo1C7NmzcJPP/1U6rnqtr0uZGRkAPjfPtCcUS4oKABwd39nZ2cDqN4+v5+IYOLEiTA2NtaqT7Vo0QITJkzAypUrsXTpUowePVp5rjr7g4iIiIiIGrn7v7hek2vG3333XXF3d5cvv/xSdu7cKXFxcZKQkKBca3z79m1xc3MTADJq1CjZsGGDzJ49W/z8/JRrky0tLeXmzZsicnfwLFtbW2WAtPbt24uVlZUkJycr27S1tRUAsmvXLtmwYYO0bNlSAMjhw4fl0qVL5S6Tk5MjAKRNmzYiIpKXlydeXl7Stm3bUteNT5kyRXr16iWFhYUiIuLi4iIAlAHOREQcHR0FgDJPdSxYsEAAyOnTp0Xk7nXUnp6e4u/vX2q+qtpNm3Z1dXUVAKJWq6WoqEh8fX2ladOmcuLECWU7lbX99u3bxdraWn766acqX5ezs3OZa7+1aTvNtd/3Xpu+Zs0a6dq1qzLPSy+9JABkzpw5cv78eVm8eLHY2dkJANm5c6cUFxeXu8/T0tIEgDg6OoparS6VNysrS8aOHasMoFdVn9K4cuWKmJmZSb9+/aq1v+5tD20HbbsXrxknIiIiImowamcAt927d4uDg0OZQatatGgh3333nYiIXLx4UQICAsTOzk5atWolY8eOLTWYGAB5/PHH5cMPP5ShQ4fKgAEDJDExUUREwsPDpXXr1sq6RESWL18utra20r17d4mPj5eoqChp1qyZDBw4UNLT08ssk5ubK+Hh4Uq2RYsWSXZ2tuTk5MiMGTPEz89PwsLCZMaMGfLuu+/KnTt3lO1ollmwYIFkZWUpg4sBkJkzZ0p+fn612uvGjRsybdq0UtO+/vpriY+PLzNvVe1W0fMZGRny3nvvKTk/+OADSU1NVQana9KkiUREREhmZmalbb97925p06aN7Nu3r8LX89tvv8nMmTOVbQ0dOlS2bt2qddtpivGPP/5Ybty4IdeuXZMPP/ywVMGakJAg3t7eYmVlJX5+fpKQkCB9+vSRYcOGybfffit37twps8/37dsnAwcOVLbn6ekpTz31lDz11FPSsWNHMTMzEwCyZs0arfrUvQYMGCDr1q3Ten/l5ubKu+++q2QZO3ZsqT+IaIPFOBERERFRgxGtErnnO7gAoqOjERwcjPsmV3ZmHd988w3S09Px1ltvAbj7tfXLly/jl19+wZtvvonr169rtS5qnDp16oSEhASt+5y+5eXl4dFHH8WpU6dgZWWls+0GBQUBuPseJSIiIiKiei2m5sNk/5/IyEi8/fbbSE9PV6YZGxvD2dkZffr0Qdu2bR90Ewbv/hHDy3Pu3Dl06tRJB2morq1YsQKvv/66TgtxIiIiIiJqWB64GD906BAAYOXKlRg3bpxy66zjx48jMjISGzZseNBNGLz6ckbXUGlGH8/Nza3RqPi6cPjwYYSGhiI/Px9qtRp///23viMREREREVE99sCjqa9duxaTJ0/GV199BScnJ/Tq1QuBgYE4ceIENmzYAC8vr9rISQ1Qbm4uZs2ahdTUVADAlClTEB8fr+dU5bO2tkZOTg6MjIywadMmmJub6zsSERERERHVYw98zTgR6QavGSciIiIiajBiauU+40RERERERESkPRbjRERERERERDrGYpyIiIiIiIhIx1iMExEREREREekYi3EiIiIiIiIiHWMxTkRERERERKRjLMaJiIiIiIiIdIzFOBEREREREZGOsRgnIiIiIiIi0jEW40REREREREQ6xmKciIiIiIiISMdYjBMRERERERHpGItxIiIiIiIiIh0zqeiJoKAgXeYgMmiXLl2CnZ0dmjRporcM8fHx6Nmzp962T0REREREtadMMe7t7Y1XXnkFarVaH3mIDE5JSQlOnz6N/Px8NGvWDM7OznB2doa1tbVOc/Ts2ROBgYE63SYREREREdUNlYiIvkMQGbqSkhL8/vvviImJwZYtW3Dt2jV07twZgYGBGD58ONq3b6/viEREREREVH/EsBgnqia1Wo34+HjExMTg22+/xY0bN9C5c2eMGDECw4cPh6Ojo74jEhERERGRYWMxTvQg1Go1fvnlF6xbtw6xsbHIzc1Vvk4eHByM1q1b6zsiEREREREZHhbjRLXlzp072LNnD2JiYrB161bk5+fjySefRGBgIIYMGQIHBwd9RyQiIiIiIsPAYpyoLuTl5WHHjh3YsmULdu7ciZKSEvj5+WHYsGF48cUXYWlpqe+IRERERESkPyzGiepaTk4Otm3bhm+//Ra7d++GlZUVBg0ahBEjRqBv374wMjLSd0QiIiIiItItFuNEunTz5k189913WLduHeLi4tC2bVsMGjQIo0aNwmOPPabveEREREREpBssxon05dy5c9iyZQvWr1+PxMREZUT2kSNHcuA3IiIiIqKGjcU4kb5p7mG+fv16bN68GXl5efD19cXw4cMxaNAgWFtb6zsiERERERHVLhbjRIZEMyL7+vXrsXXrVpiYmMDf3x/Dhw/HCy+8AGNjY31HJCIiIiKiB8dinMhQXbt2DZs3b8a6detw4sQJODs7Y/jw4Rg1ahQ6dOig73hERERERFRzLMaJ6oOzZ89i/fr1WL9+PS5fvoy+ffti9OjRGDx4MKysrPQdj4iIiIiIqofFOFF9UlJSgv3792PVqlWIjY2FhYUFXnzxRYwYMQL9+/fXdzwiIiIiItIOi3Gi+iozMxMxMTH4/PPPcfLkSXTq1AmvvfYaRo0aBQcHB33HIyIiIiKiirEYJ2oIjh8/jnXr1mHDhg3Izs6Gr68vQkND8Z///AcmJib6jkdERERERKWxGCdqSPLz8/Hdd9/h66+/xm+//QZHR0eMHDkSISEhcHd313c8IiIiIiK6i8U4UUP1zz//4JtvvsGaNWtw9epV9O/fH2PHjsWLL74IU1NTfccjIiIiImrMWIwTNXT3Dvr2//7f/0Pz5s3x2muvYezYsWjfvr2+4xERERERNUYsxokak8uXL2P9+vX4/PPPcenSJTzzzDMYPnw4AgMDYWlpqe94RERERESNBYtxosbo/rPlTZo0QWBgIKZMmQIvLy99xyMiIiIiauhYjBM1dpqz5StXrkRSUhKeeOIJhIaGYvjw4TxbTkRERERUN1iME9FdarUaO3fuxKpVq/DTTz+hefPmCAkJwbhx4+Di4qLveEREREREDQmLcSIqKy0tDatXr8bq1atx7do1+Pv7Y+LEifDz84NKpdJ3PCIiIiKi+s6wivGkpCQcPXpU3zGIKtW6dWv4+PjoO4ZOqNVq/PTTT1i6dCn27duH9u3bY8yYMRgzZgyaN2+u73hERERERPWVYRXjQ4YMwebNm/Udg6hSJiYmKCoq0ncMnTt//jy+/vprrFq1Cvn5+Rg4cCCmTZuGnj176jsaEREREVF9Y1jFeFBQEAAgOjpaz0mIyhcdHY3g4GAY0NtG527duoVvv/0WK1aswKlTpzjgGxERERFR9cUY6TsBEdUvTZo0QWhoKP78808cO3YMnTt3xuTJk+Ho6Ig33ngDFy9e1HdEIiIiIiKDx2KciGrsiSeewLp163Dp0iWEh4dj69at6NChA/z8/BATEwO1Wq3viEREREREBonFOBE9sNatW2PmzJlITEzE1q1bAQDBwcHw9PREZGQk0tPT9ZyQiIiIiMiwsBgnolpjbGyMgIAA7NmzB3///TcGDx6MyMhIODk5ISgoCL///ru+IxIRERERGQQW40RUJzp27IiPPvoIycnJWLp0KRISEtC7d29069ZNGZGdiIiIiKixYjFORHWqogHf2rZtywHfiIiIiKjRYjFORDqjGfAtOTkZ06ZNw/fff48OHTogICAAu3btatS3jCMiIiKixoXFOBHpXJs2bTBv3jwkJSUhOjoa+fn5eO655+Dl5YWVK1ciLy9P3xGJiIiIiOoUi3Ei0hsTExMMGjQI+/btQ0JCAvz8/DB9+nTlnuXJycn6jkhEREREVCdYjBORQejYsSOWLFmCy5cv47333sPWrVvh7u6OgIAA7N27V9/xiIiIiIhqFYtxIjIoTZs2xRtvvKHcszwzMxN+fn7o2rUrVq1ahdu3b+s7IhERERHRA2MxTkQGSXPP8kOHDuHYsWPo0qULJk+eDFdXV4SHhyMtLU3fEYmIiIiIaozFOBEZPM0o7JcuXcKECRPw1Vdfwd3dHUFBQTh8+LC+4xERERERVRuLcSKqN1q3bo358+cjNTUVq1evRkJCAnr27Ilu3bph3bp1KC4u1ndEIiIiIiKtsBjXsWvXriE6OhoffPCBvqMQ1Vvm5uYYMWIE/vzzTxw8eBDu7u4YPXo0XFxcMH/+fNy8eVPfEYmIiIiIKtUgivEePXpgxowZ+o5RpXPnzuG9995DcHAw1q9fX6vrPnjwIMLDw6FSqaBSqTBixAjExsbW6jZq4pdffkFgYKCSa9y4cYiLi9N3LGpA+vTpg+joaJw/fx7Dhw/H0qVL4eLignHjxuHs2bP6jkdEREREVC6ViIi+Q2gEBQUBAKKjo6u13CuvvAIPDw8sWLCgLmLVqjt37sDS0hKenp74+++/a339Li4uuHTpEvLy8mBlZVXr69dGSkoKnJ2dlZ/z8/NhbW2Ndu3a1fv7RkdHRyM4OBgG9Lah+9y6dQvffvstPv30UyQkJOCZZ57BlClT4O/vD5VKpe94REREREQAENMgzoxv3ry5XhTiAGBhYVGn67e0tAQAvRXiFy9exNChQ0tN02TRZCOqS02aNEFoaCj++usv7N69GxYWFnjxxRfh6emJJUuWIC8vT98RiYiIiIgaxtfUyTCkpqbC398fN27c0HcUIhgZGaF///7Yvn07Tp48ib59+yI8PBzt2rXDrFmzcPnyZX1HJCIiIqJGrF4X42q1GtHR0Rg5ciT69u2rTD969Ch69OiBSZMmYe7cuTAxMUFubq5W68zLy8OGDRswZMgQ9OrVC/Hx8Xj88cfh4uKCQ4cOISEhAS+99BJatGiBTp064dixY6WWP3/+PAYPHoyZM2di+PDh8PHxwalTpyrd5u3btxEZGYmQkBB069YN/fv3x+nTp5Xn9+/fDycnJxw4cKAarQOICGJjYxEaGgonJydkZmZi5MiRaN68Obp06YJjx45BRBAfH4/p06fD1dUVV69exaBBg2Bvb48uXbrg+++/BwCsWrVKue4bAHJycrBo0aJS09asWYOzZ8/i6tWrGD9+fLWyalTWfhs2bICVlRVUKhU++ugjZeTsjRs3wszMDGvWrKm0PdVqNX799VdMnToVrq6uSEtLQ79+/dCuXTtkZmbWKC/VD4888gi+/PJLXLp0CdOmTcM333wDNzc3vPbaa1W+P4mIiIiI6oQYkMDAQAkMDKzWMsnJyQJAPD09lWkeHh5iZ2cnJSUlIiISFBQk165d02p9arVaLly4IADE1tZWduzYIX/99ZcAEBcXF1m4cKFkZWXJiRMnBID069ev1PIdOnQQd3d3EREpLCyUpk2bipeXV6l57s87ZswYOXfunPKzn5+fODg4SHZ2toiIbN26VSwtLWXbtm1V5vf09BTNbi0pKZGUlBSxtrYWAPL+++9LUlKSrF+/XgCIt7e3FBcXy/bt28XCwkIAyOTJk+XAgQOyceNGsbGxEQBy6NAhERFxd3eX+7vM/dPuf21VTb9fVe03e/ZsASBnzpxRpiUnJ8tLL72k/FxRe16/fl3i4uLE0tJSAEhERITs2bNHQkJC5NatW1VmExHZsmVLmTag+qegoEDWrl0rXbp0EQDSu3dv2bZtm/KZQURERERUx6INqqqoSTFeUlJSptBr0aKFAJCoqChRq9Vy+vRppbCt6TodHR1LFWElJSXSsmVLadq0aallFy1aJJs2bRKRu4W9u7u7mJiYlJrn3nUfPnxYAJT72L59u7JMUVGRVtnvLcY1OnbsWCa7g4ODmJmZKdM8PDwEgOTm5irTPv30UwEgwcHBFa77/mkPWoxX1X7p6eliY2MjY8aMUaZFREQobaVNe2raIyMjo8o892Mx3vAcPHhQ/P39RaVSSceOHSUqKkry8/P1HYuIiIiIGrboev01dQDljo78+eefw8bGBlOnToW3tzdyc3Nha2v7QOts0qRJmXns7e2RnZ1danpYWBgCAgKwYsUKfPDBBygoKFC+Tl2eo0ePwsvLCyJS5uHv76/MZ2JionX+ql6PSqWCnZ0dCgsLlWlGRne7grW1tTJt4MCBAIALFy7UeNvVVVX7NW/eHK+//jrWrl2LtLQ0iAj27duH5557DoB27alpD3t7e529LjJcffr0Ua4rf+qppxAeHg5XV1fMnz8fGRkZ+o5HRERERA1UvS/GyzN48GCcPHkSzz77LI4fPw4fHx/leuK6duTIETz88MNwd3fH3LlzYWNjU+n8GRkZSExMLHeEZ7VaXVcxteLo6AgApW5TVleuX7+OoqIirdovLCwMZmZmiIqKwvHjx+Ht7a38scKQ25MM2yOPPIIvvvgCSUlJmDBhApYtW4a2bdtixIgRdXIbQiIiIiJq3BpkMT5v3jy0b98eu3btwqZNm1BcXIw5c+boZNsjRoxAUVERnn/+eQBASUkJAFR4X+pOnTopA47d6+zZs1i+fLnyc2Vn1+uK5qxg//79AfzvjHJBQQGAu69N882Ae19fdbOKCCZOnAhjY2Ot2q9FixaYMGECVq5ciaVLl2L06NHKc9q2J1FFWrVqhfnz5yM5ORlLly7FkSNH4OXlhYCAAOzdu1ff8YiIiIiogaj3xfitW7cA3B3dW+OTTz5RRscePHgwbG1t0bZtW63Xefv2bQClC8CioqJS2wOAO3fuACh9xvXKlStIS0vD7t27sXHjRmRlZQG4e8Y8JSUF+fn5pZZ98cUX4ebmhgULFmD06NHYuHEj5syZg6lTp2LUqFEAgB07dqBZs2bYuXNnldk167/3zLBmW/e+Hs3r0LwujXsL6b1796Jr164YN24cgLuFLgC8//77uHDhApYsWaIU5rt27YJarUb79u1x5coVXLp0SVmP5hZSt27dUoprjezsbIwbNw4WFhYwMjKqsv00pk+fjsLCQly6dAkdOnRQpmvTnpr20HaEfWqcbGxsEBoairNnz2Lr1q3IzMyEn58fnnjiCaxbt67KPzrduXMH/v7+2L17t44SExEREVF9Uq+L8by8PERERAC4WwQvXrwYOTk5uH37Np555hl89NFHeO211+Dj44PNmzdrtc5r165h9uzZAICkpCTs3bsXu3btQnJyMgBg9uzZyMjIwLJly5RpixYtQnp6OgAgIiICtra2mDNnDtq3b4/Zs2ejWbNmiIiIwNWrVxEeHg4ASE5ORlRUFG7fvo39+/cjICAAW7duxfTp03H9+nVs3LhRuc7d3Nwctra2MDc3rzD3wYMHER4erhSsoaGhiI2NxYoVK5ScH3zwAbKzsxEVFaUUyHPnzlX++AAAUVFRSE9Px/Xr13HlyhUcOHAApqamAIDIyEh4e3tj8eLFmDRpEgYMGAAvLy8MGzYMWVlZKC4uRmBgIGxtbXH06FEAd2/LNmHCBAB3i/LOnTvD19cXvr6+8PT0hIODA1avXg0/P78q28/KykrJ2bp1a/j5+SEkJKRUO1hYWFTYnsbGxnjvvfeU9ggLC8Mff/yhVb+gxsvIyAgBAQE4dOgQjh07Bi8vL4SEhKBjx46IjIxU/mB0v/Xr1+PHH3/EgAEDsG3bNh2nJiIiIiJDp5KKvj+tB0FBQQCA6OhoPSdpfDp16oSEhIQKv05vaPLy8vDoo4/i1KlTpYr0uhYdHTvPYFgAACAASURBVI3g4OB6005UNxITE7FkyRJ89dVXMDY2xmuvvYY333xTGV+hpKQEnp6e+PfffwHcvcRj7dq1GDZsmD5jExEREZHhiKnXZ8arS6VSVfngQE31w4oVK/D666/rtBAn0nB3d8eSJUtw+fJlvPfee/jhhx/Qvn17BAUF4ciRI9i6dSv++ecfZST/kpISjBw5UmcDSRIRERGR4av5/bLqIZ7NrJjmGvPc3NwqR4DXl8OHDyM0NBT5+flQq9X8wwnpna2tLd544w2MHz8emzZtwuLFi9GjRw+0aNECxsbGpcaTKCkpwejRo5Gbm4vJkyfrMTURERERGYJGdWacysrNzcWsWbOQmpoKAJgyZQri4+P1nKp81tbWyMnJgZGRETZt2lTpNfREumRubo5Ro0bh1KlT+OSTT5Cenl7urfREBFOmTEFUVJQeUhIRERGRIWlUZ8apLBsbG0RERCgD4Rmyhx9+GElJSfqOQVQhlUqF/fv3w9TUtMydCjREBNOmTcOtW7cwd+5cHSckIiIiIkPBM+NENXDs2DHlPuxEGgkJCdi5c2eFhfi93nnnHeXuCkRERETU+PDMOFENdO/eHcDdW6k5OjrC3d29zKNTp06wtrbWc1LSpYULF8LIyKjcr6jfT0SwcOFCiAgiIyN1kI6IiIiIDAmLcaIa+Pfff5GYmIjExERcvnwZV65cQWJiIvbu3YvExERlPjs7u3ILdXd3d7i6usLIiF9OaUhOnDhRqhBXqVQwNTWFSqVCcXFxmSJdU5Dfvn0bS5YsgUql0nVkIiIiItITFuNENaApqMtz584dXL58WSnWNY+9e/fi77//VkauNzc3R9u2bcst1D08PGBra6vLl1TnwsLC8Omnn+o7hk6JCAoLC6ucb9myZVi2bJkOEhHVLicnJ6SkpOg7Ro0cPHgQTz/9NIqLi/UdpVGaNm0aFi9erO8YNdIYj2eGwsTEBPv374ePj4++o9SIs7OzMmgyNS4VHS9ZjBPVMgsLi0qL9czMzDKFuqZYT0pKQklJCYDKz6q7uLjA2NhYly+rlDNnzuDWrVvo2bOn1sukpqbiySefRFhYWB0mqz/UajVycnKQlZWFnJwceHh4wMrKSt+xiLQWHx9frwuSK1euoLi4GNHR0fqO0ugsXry4XhckPJ7pT1BQEK5cuaLvGDWWmpqKadOmVev3J6r/Kjteshgn0jE7Ozs88cQTeOKJJ8o8V1BQgLS0tHIL9QsXLiAnJwcAYGpqCmdn5wqLdTs7uzp9DTNmzMDPP/8MX19ffPDBB1ofVJydnREYGFin2YhIN0RE3xFqBT+TdC8mJkbfER4Yj2dUU08++ST7TiNT2fGSxTiRATE3N6/xWfXk5GTlmuTKBpZ76KGHHvgM7Pnz5wHc/Zpnr1690L9/f7z//vvo0aPHA62XiIiIiKixYDFOVI9Udla9qKgIKSkpZQr148ePIyYmBpmZmaXWU9FZdTc3t0oHEhMR5euFmmstDxw4gCeffBJPPfUUFi5cqIw2T0RERERE5WMxTtRAmJqaVnlW/d6R3+89q37u3Dnk5+cDqHxguY4dOyInJ6fMoGSa+2rHxcXB29sbTz31FD7++GN069atbl80EREREVE9xWKcqJGws7ODnZ0dvLy8yjynVquRmpqKxMREXLx4Ufn31KlT2Lp1K65fv67M26pVqwq3oSnKDx06hO7du8PX1xcff/xxuWfyiYiIiIgaMxbjRARjY2O4uLjAxcUFvr6+ZZ7Py8tTCvTY2FisWbNGGfW9PJqvrx88eBDdu3fHc889h6KiojofWI6IiIiIqL5gMU5EVbK2tsbDDz+Mhx9+GH/88QdMTU1RUFBQ5XKaonznzp1QqVTw8/Or66hERERERPUCi3EiqpbExESlyC6PmZkZiouLUVJSAktLS3Tt2hW9evVCXFwcbG1tdZiUiIiIiMhwGVwxnpKS0iDuP0kN0+HDh/UdQe8SEhKUW6gZGxsDuHvNuYmJCR566CH07dsX3bt3R/fu3dGpUycYGRkBAIKCgvSWmYiIiIjI0BhUMe7k5ISYmBj+0k4GzcnJSd8R9CorKwsqlQrt27dH79694e3tje7du+PRRx+FmZmZvuMREREREdULBlWML168GIsXL9Z3DCKqRFxcHMzMzNCkSRN9RyEiIiIiqrcMqhgnIsPXvHlzfUcgIiIiIqr3jPQdgIiIiIiIiKixYTFOREREREREpGMsxomIiIiIiIh0jMU4EdU7WVlZ+o5ARI1Udna2viNQPcW+Q/Ud+3DtYzFORPXCnTt38MEHH6Bnz571chA5EcHSpUsxc+ZM+Pr6wsfHBwkJCfqOpVN79uzB888/D5VKBZVKBV9fX/j6+qJbt24YOHAgvvzySxQUFOg7Zikff/wxmjVrBpVKBWNjYzz77LPw9/fHgAED8Mwzz6Bdu3ZQqVS4dOmSvqNSHfv444/Rt29frT5/0tLS8PXXXyMoKAg9e/as0fbq4/uFyqdt3xERfPnll3jsscdgY2ODRx99FF9//TVEpFrbY9+hqvTo0QMzZszQev7qfP7VRKPus0JEpAOBgYESGBj4QOvIz88XOzs7qY8fXVFRUWJtbS1FRUWSmZkp//nPf+S///3vA6/30qVLtZBOd1JTUwWAuLq6KtPUarVs27ZN3N3dpUOHDnLmzBk9JiwrLS1NAEiHDh3KPKdWq2XAgAHyzz//1Mq26tP+3LJli8G8Fzdv3ixvv/22nDp1Sutlqpv/9u3bYm9vr/UyycnJAkA8PT213sb96uP7RRu1cTyoDTk5ORIaGiqbNm2S3NxcrZerbn5t+87MmTPl1VdfleXLl8uUKVPEwsJCAMjSpUu13pZGQ+07AGTLli36jiEnT56UiRMnyt69e6W4uFjr5Qwlf3BwsMyZM0fr+avz+VfT41hD7bMilR5vonlmnIjqDUtLSzg4OOg7Ro18/vnnaNu2LUxMTNCsWTP88MMP8Pb2fqB1Xrx4EUOHDq2lhLrRtm1bAIC5ubkyzcjICAEBATh48CByc3MxcOBA3L59W18Ry2jTpg0AwNjYuMxzRkZGePvtt2FjY/PA26mP+9NQREdH48MPP8QjjzwCT09PRERE4OLFi7W6DQsLC7Rs2VLr+du1a/fA26yP75f6JDExEatWrcLQoUPRvHlzvPLKK9ixYwcKCwtrdTva9J2UlBSkpKRgw4YNmDRpEpYsWYKtW7cCAJYsWVLtbbLv1K0DBw7gs88+Q//+/dGqVStMnToV//3vf/UdS2ubN2/GggULtJ5f28+/BzmONdY+y2KciEgHUlJSoFKpam19qamp8Pf3x40bN2ptnfrm6OiIBQsWIDExEYsWLdJ3HEVl++3kyZPo3bs3WrVq9UDbaIj7U1/Onz+P+fPnw93dHZ6enoiMjMTVq1f1HavWGer7pb4qKCjADz/8gIEDB8Le3h7Dhw/H9u3bUVxcrJPtJycnl9mPzz77LFq0aIHr16/X6rbYd2qHmZkZACAjIwOfffYZnnzySTg6OuKNN97AH3/8oed0uleXx7GG3GdZjBORwcrPz0dYWBhCQ0MxZ84cvP3228jLyys1z+3btxEZGYmQkBB069YN/fv3x+nTpyEiiI2NRWhoKJycnJCZmYmRI0eiefPm6NKlC44dO6as4+jRo+jRowcmTZqEuXPnwsTEBLm5uZWuX1s7duzA+PHjkZ+fj6tXr2L8+PEYP348cnNzq1z3+fPnMXjwYMycORPDhw+Hj48PTp06BQBYs2YNzp49q6wTAFatWqVcbwUAOTk5WLRokTJNrVbj119/xdSpU+Hq6oq0tDT069cP7dq1Q2ZmZpV5Kmun/fv3w8nJCQcOHKjOLi5j8ODBMDIywu7du5VphriPCwsLcfr0abz++utV5tSoT/uzvisqKgIAXLhwAbNnz4ajoyN69uyJJUuWID09/YHXf/36dQwaNAj29vbw8vLC0aNHq7V8Y3u/1BdFRUUQEeTl5SE6OhoDBw5UiqtDhw5V+9rt8lTUd/r06YPWrVuXmb+wsBA+Pj7Kz+w7hknzmXPlyhWsXLkSXbt2hYeHB+bPn49///23Ruvcv38/zMzMYGNjg99++w1ZWVkYNmwYVCoVnnrqKZw5cwYAcOLECbRp0wZffPEFgMrbWa1WIzo6GiNHjkTfvn2VbYkIli1bhmHDhmHChAkwNzdXjjX3/0G6oj5c3nFM8zrYZyuhq+/KE1HjVt1r7IqKisTb21vGjBkjJSUlIiLyzz//iLGxcanrbsaMGSPnzp1Tfvbz8xMHBwfJysqSlJQUsba2FgDy/vvvS1JSkqxfv14AiLe3t7KMh4eH2NnZKdsJCgqSa9euVbr+7Ozsar1+lHPtaFXr7tChg7i7u4uISGFhoTRt2lS8vLwqXae7u3uZ65I00+7cuSNxcXFiaWkpACQiIkL27NkjISEhcuvWrSrzVNZOW7duFUtLS9m2bVuN2uJerVu3Fnt7+yrbSZf7GEC5j6ZNm1aZsz7uT20Y0jXjL7/8coX7SPMwNjYWY2NjMTExkeeff14mTZpU7fyenp4CQObNmycXL16UHTt2CAB58skny52/or7e0N8vVTGUa8ZPnjxZZb8BIKampgJA2rZtKzNnzpTnnnuu2vmr23dERA4dOiQWFhZy/PhxZVpj7zswkGuulyxZImZmZpX2G5VKpfSdRx99VKKioqqdf8KECWJubi5ZWVkicnfsHAcHB3n11VeVeYqKisTHx0dp86raubzxLJYuXSpGRkaSnp4uIiIRERECQMLCwpR5tOnD5fW5xt5nRSq/ZtwwjqJE1OBV95evZcuWCQA5e/ZsqekeHh7KB9rhw4crPAhu375dREQ6duxY6gOwpKREHBwcxMzMTJnWokULASBRUVGiVqvl9OnTkp2drdX6tXX/AUabdS9atEg2bdokIncHMXF3dxcTE5MK1ynyv4NlZdM0bZKRkVGtPBW1k0ZRUVGN2uJ+Tk5O0qZNG61z6WIf35+5uLhYzp8/L4888ojWOevb/qyK5pcLQ3i0bdu2xstWh6bt1Wq1iNzta/b29mJpaVnu/JX19Yb8fqlKYGCgso36+rj3l3ltVLfvFBUVSd++fZXPjPuf00ZD7DsAlD9A6vuhKbS1eahUKuX/1RmQ76+//hIAsmLFCmVaQECAWFtbS05OjoiIxMbGysqVK7XejyUlJWX6RkBAgKhUKikoKBARkdOnTwsA6dGjhzKPNn24oj7XmPusSOXFuAmIiAyQ5mtIrq6upaYbGf3v6pqjR4/Cy8tL+apWee7/epVKpYKdnV2pa/A+//xzjBo1ClOnTsX69euxfPly2NraarX+mtJm3WFhYcjNzcWKFStw8+ZNFBQU1Mr1i5o2sbe3r1aeitpJw8TkwQ8phYWFuHbtGvr37691Ln3sY2NjY3h4eGDSpEla56xv+1Nb0dHR1Q9dyz777DOkpaVVOZ/m88PExARdu3bF4cOHa7Q9zXpUKhVatmxZo9sUNqb3S3m8vLyU94++JCcna317J1NTUxQVFaFDhw6wsLBQBnasLm37zrvvvotnnnkGQ4YMKfNcY+87EydORI8ePR54PQ9i586d2LhxY5XzaW6LqVar0bt3b8TFxVVrINrOnTvD19cXX3zxBSZMmIDk5GSo1WoUFhbi22+/RWhoKNatW4e1a9cCqNl+BAA/Pz9s374dP/74I/7zn//AwsICAPD000+Xmbcmn3+Nvc9WhsU4ERkkzS/WGRkZcHJyKneejIwMJCYmIi8vD9bW1qWeU6vV5Y5+XZ7Bgwfj8ccfx8SJE7F79274+Phg9erVtbb+mmY/cuQIgoOD8dlnn2HSpElaHfjrMk9F7fTaa6/VWo79+/ejqKgIzzzzjNa5tFFX+zg0NFTrnA11fwYGBtZi8prZvHlzhc+pVCoYGRlBRODt7Y1Ro0bhlVdewc8//4zg4GAdpqx99e39ci8HBwe9950///yz0ufNzMxQWFiIli1bYsiQIQgMDESfPn0QFBRUp7m2b98Oa2trhIeH19k26nPf8fb21nvfuXLlSqWf4SYmJiguLkaHDh0wdOhQjBw5Em5ubuVeg12VyZMnY9CgQTh69Ci++eYbLFy4EAsWLMDq1avRu3dvuLq6Km1a03aePHkyLC0tERISgri4OFy4cAHvvvsuZs2aVa2sdak+99nKcAA3IjJInTp1AgD8+OOPlc6jGVzjXmfPnsXy5cu13ta8efPQvn177Nq1C5s2bUJxcTHmzJlTa+uvafYRI0agqKgIzz//PACgpKQEAEoNInT/mVXNQb6goEBZJjs7u8xyNclTUTtVlKW6CgoKMGvWLDz22GOYMmWK1rm08SD7uLJ202iI+7M+U6lUMDU1BQA88sgjWLRoES5fvoz4+HiEhobW6BsAta2hvl/qO80ZvCZNmiA4OBh79uzBtWvXsGTJEvTp06fOt797926kpqaWKcR///135f/sO4ZJM7p627ZtMX36dPz999/K3R3c3NxqvN6BAwfCyckJ8+fPR15eHry8vDB+/HgcO3YMEydOxIQJE5R5a9rOarUaZ86cweHDh/HJJ58gNjYW8+bNq9EZ7fL6J/tsJar1hXciohqq7jXjf/zxhxgbG4u9vb3s3LlT8vLyZN++fdKkSRMBIImJiXL79m1xc3MTADJq1CjZsGGDzJ49W/z8/JRrX11cXASAMjCHiIijo6MAkMLCQhERsbS0lJs3b4rI3YG1bG1txdvbW6v1ayMjI0MAiJubmzJNm3Xb2toKANm1a5ds2LBBWrZsKQDk8OHDcunSJWnfvr1YWVlJcnKyst6XXnpJAMicOXPk/PnzsnjxYrGzsxMAsnPnTikuLlba5NatW9XKU1E7iYhs375drK2t5aeffqq0LfLy8gSAuLi4lJp+/Phx8fHxEVdXV/nrr7+qlauu9/GtW7cEgDg7O1f4uhra/tSGIQ7gZmJiIgCkc+fOsnDhwlJteb+a5G/Tpo0AKPX+b926dZn2F/lfX+/QoUOZ9TTk94s2DG0ANyMjI1GpVGJpaSnDhg2TnTt3VnqNa03ya9N39uzZI76+vrJs2TLlsXTpUpk6darMnj1bRNh3AMMZwM3IyEi5brxly5Yyffr0UoPtlaem+RcsWCAA5PTp0yJy99pnT09P8ff3LzWfNu2ck5MjAJRrr0VE3n33XXF3d5cvv/xSdu7cKXFxcZKQkFDqfaBNHy7vONbY+6wIB3AjIgNQk19eDhw4IL169RIbGxtxc3OTDz/8UHx8fGTcuHGyd+9eKS4ulosXL0pAQIDY2dlJq1atZOzYsXL9+nUREVm+fLky2MaCBQskKytLPv30U2XazJkzJT8/XwDI448/Lh9++KEMHTpUBgwYIImJiSIila5fG6dOnZJx48YpA7jMnz9fTp48qdW6ly9fLra2ttK9e3eJj4+XqKgoadasmQwcOFDS09MlPDxcWrduLd99952yTEJCgnh7e4uVlZX4+flJQkKC9OnTR4YNGyZffvmlzJo1S3n9Y8eOlRMnTijLVpWnsnbavXu3tGnTRvbt21dhWxw8eFBGjx6tbL9fv37y7LPPSkBAgLz88suyfPnyMgVNVbnqeh/HxcWVyjxz5swKf9lqSPtTG4ZUjL/xxhvSrl07mTt3bplBHytSnfxqtVoWLlyotPUbb7wht27dksjISGVaWFiY3LlzR0RE9u/fL2PHjlX+QBAZGSl//PGHsr6G+n7RlqEU4ykpKcp7MDo6WvLz87Varjr5te07+/fvr3Rgsn/++UdE2HcMpRj/7rvvxN7eXkJDQ+XXX39VBjWrSk3z37hxQ6ZNm1Zq2tdffy3x8fFl5q2snXNzcyU8PFzZb4sWLZLs7GzZvXu3ODg4lOl3LVq0kOjoaK0//8o7jjX2PitSeTGuEqmFmyYSEVVBc42dIQz2REQPLjo6GsHBwbVy72V9qO/567P6fjyo7/nrM5VKhS1bttT5dft1xRDziwi++eYbpKen46233gJw92vrly9fxi+//II333yz1IBnVH2VHG9iOIAbEVENaTMIy7lz55Tr34mIiIgMSWRkJN5++22kp6cr04yNjeHs7Iw+ffqgbdu2ekzX8HEANyKiGhKRKh8sxImIiMhQHTp0CACwcuXKUgX58ePHER4ejg0bNugrWqPAYpyIiIiIiKgRWrt2LSZPnoyvvvoKTk5O6NWrFwIDA3HixAls2LABXl5e+o7YoPFr6kRERERERI1Q8+bNsWzZMixbtkzfURolnhknIiIiIiIi0rFaPzN+8OBBPP300w98c3eihmjatGlYvHixvmM8kF9++QWJiYnw9vZG586dYWxsrO9IRERERET1Tq0X41euXEFxcTFv90B0n8WLFyM1NVXfMR7YokWL8OOPPwIALC0t0bVrV/Tu3Rs9evSAt7c3nJyc9JyQiIiIiMjw1dk144GBgXW1aqJ6KSYmRt8RaoWrqytMTU1RVFSE27dvIy4uDkeOHEFxcTFEBM2bN4e3tze8vb3xxBNPoHfv3rC3t9d3bCIiIiIig8IB3IioWtzc3MrcX7uoqEj5f0ZGBn7++Wfs2bMHxcXFUKlU6NChA4qKijgiJxERERHR/+EAbkRULa6urqWK7/KIiDJuhIjgwoULSEtLw507d3QRkYiIiIjI4PHMOBGVq6SkBJcvX0ZSUhIuXryIixcvIikpCadPn4aIaLUOExMTWFlZYdq0afjzzz9hampax6mJiIiIiOoHFuNEjdjVq1eVIltTdGv+n5ycjMLCQgCAubk5XFxc4ObmBi8vLxw7dqzS9d5bhE+bNg1NmzZFUFCQLl4SEREREVG9wGKcqAHLzMxEYmIiLl++jCtXriAxMVF5nD9/Hrdu3QIAmJqaokWLFnB0dIS7uztefvlluLu7Kw8XF5dStzCLiYlBfn5+me2VV4QTEREREVFZLMaJ6rE7d+7g8uXLpYpszeOff/5Bdna2Mq+dnZ1SXPfv3x+hoaHKz+3atYOJifYfB87OzkhISFB+ZhFORERERFQ9LMaJDFhBQQHS0tLKLbY1Z7s1yiu227RpA0dHRzz00EOwsrKqtVwdO3ZEQkICjI2NYWNjg5kzZ2Ly5Mlo0qRJrW2DiIiIiKghYzFOpEeFhYVITU0tt8hOTEzExYsXlcHS7i22e/furXyl3N3dHZ6enrCxsdFZ7ocffhgHDx5UinBdbpuIiIiIqCFgMU6kQ4mJiXj11VeVwdLuP7Pt5uYGV1dXPPLIIxg4cCDc3NyUadbW1npMXtqCBQvwzjvvwMzMTN9RiIiIiIjqJRbjRDqUl5eH3NxcdOvWDYGBgXB1dVWK7fp0nbWRkVG1C3FjY2Ns3rwZKpWqjlIRka5VZ6wJQ6PJzs8k/XjllVf0HaHGeDzTr/r+uRMcHIzg4GB9RyEdq6jf1t/eTFQPPfzww4iOjtZ3DL348MMP8fLLL+s7Rq3bsmULfv75Z+Tn55e5/7qRkRGMjIygUqlQUlICtVqtPGdhYYHZs2fD09NT15GJak3r1q31HaHGXnjhBXz//fel3pekO927d9d3hBprqMez+sDY2BgvvPCCvmPU2P79+3H16lV9xyA9qOh4yWKciHTC1dUVrq6u+o5R65KSkvD999+X+1xJSQlKSkpKTTMxMYGtrS327duHxx57TBcRiagcFhYWLKioRhrq8Yzqno+Pj74jkIEx0ncAIqL6bOzYsbCwsNBqXlNTU7Rs2RLx8fEsxImIiIgaORbjREQPoFmzZggJCYGpqWml85mamqJdu3b473//i44dO+ooHREREREZqkZdjGdnZ+s7AhE1AEOGDEFxcXGFz5uamuKhhx5CfHw8nJ2ddZiMiIiIiAxVoyzGP/74Y/Tt2xfNmzfXdxS9O3jwIMLDw6FSqaBSqTBixAjExsbqOxZ++eUXBAYGKrnGjRuHuLg4fcciKiU+Ph5DhgyBr68vWrduXe7ZcRMTE3Tr1g0HDx5Ey5Yt9ZCSiIiIiAyRSu4f/vcBRUdHIzg4uMyowvqWkpKinJG6c+cO2rZti5s3bxpcTn1xcXHBpUuXkJeXBysrK71kuHcfAUB+fj6sra3Rrl07JCcn6yVTbQoKCgKARjuaekNRWFiI2NhYREVF4ffff0fXrl0xbtw4tG3bFv7+/qXmNTY2hp+fH3744QdYWlrqKTERERERGaCYRnFm/OLFixg6dKjys4WFBc9Q3UdTKOirEL9/H92bhUUMGYJr164hMjIS7du3x5AhQ2Bvb489e/bg+PHjCA0NxYABA/DQQw/ByOjux6qxsTECAwOxbds29mEiIiIiKqPBF+Opqanw9/fHjRs39B2FKsB9RIbs+PHjGDduHFxdXbFo0SK8+uqrSExMxPbt29G/f/9S84aFhQEAVCoVxo8fj40bN1Y5sBsRERERNU56L8bz8vKwYcMGDBkyBL169UJ8fDwef/xxuLi44NChQ0hISMBLL72EFi1aoFOnTjh27Fip5bOzs/HWW28hPDwcYWFhePbZZxEWFobMzEwAwJo1a3D27FlcvXoV48ePL7P969evY9CgQbC3t4eXlxeOHj2qPHf79m1ERkYiJCQE3bp1Q//+/XH69Gmo1Wr8+uuvmDp1KlxdXZGWloZ+/fqhXbt2ynarcv78eQwePBgzZ87E8OHD4ePjg1OnTinPiwiWLVuGYcOGYcKECTA3N1eun1apVJXm09i/fz+cnJxw4MAB7XfI/207NjYWoaGhcHJyQmZmJkaOHInmzZujS5cuOHbsGEQE8fHxmD59OlxdXXH16lWlHbt06aLcd3nVqlWlMufk5GDRokWlplW1jx60PTds2AArKyuoVCp89NFHykBbGzduhJmZGdasWVNpe9bG/qb6paioCDExMfDz80O3bt1w5MgRLFmyBElJSfjoo4/Qrl27cpd79dVX4eLigrlz52L58uXKWXIiIiIiojKklm3ZJcbUhAAAGoRJREFUskWqs1q1Wi0XLlwQAGJrays7duyQv/76SwCIi4uLLFy4ULKysuTEiRMCQPr166csm5OTIx4eHvLOO+8o065duyYeHh7i5uYmmZmZIiICQDw9PUtt19PTUwDIvHnz5OLFi7Jjxw4BIE8++aQyz5gxY+TcuXPKz35+fuLg4CDXr1+XuLg4sbS0FAASEREhe/bskZCQELl165ZWr7tDhw7i7u4uIiKFhYXStGlT8fLyUp5funSpGBkZSXp6uoiIRERECAAJCwurMl92draIiGzdulUsLS1l27ZtVebRtIeISElJiaSkpIi1tbUAkPfff1+SkpJk/fr1AkC8vb2luLhYtm/fLhYWFgJAJk+eLAcOHJCNGzeKjY2NAJBDhw6JiIi7u3uZPnH/tPL2UWXT71dVe86ePVsAyJkzZ5RpycnJ8tJLLyk/1+X+FhEJDAyUwMBArecn3bt27Zp89NFH4uzsLEZGRuLv7y979uzRdywiIiIianii9V6Mi9wt/u4vuhwdHUutp6SkRFq2bClNmzZVps2aNUsAyOXLl0utb+3atQJAZsyYISKVF+NqtVpZv729vVhaWoqIyOHDhwVAuY/t27eLiEjHjh0FgGRkZFTr9YqILFr0/9u796As6/z/4y8ESUXJU2imqajBaocd3Wystm8e2MlR0HbjkFu4iofVPJSmuZ1mSyWp9SyTYZTKQb3BUdTaTcXWMnFqsd1qPWSrCYKGkIBIHITP7w/mvn/gCVC4L259Pmachs993df1vt7XMNOLz3V9rsUmMTHRGFP1BwlfX1/j4eHh+DwwMNC4ubmZ0tJSY4wx3377rZFkHnrooTrXZ4wx5eXldaqnehi3s5+fXWVlpfHx8TGenp6OsT59+hhJpqioyDG2dOlSI8mEhoZedd+Xjt1oGK+tn7m5uaZ169ZmwoQJjrHIyEhHrxr7ehtDGG/KDh48aCZNmmRatmxpbr/9djNjxgzz448/Wl0WAAAAbl62JnEPpf125eratGlz2Tbt27ev8W5w+6uuLt32sccekyTt37+/1mPbbyN1c3PTHXfcoV9++UWS9NVXX6lfv34yxlz2z75isr3u9u3b1+k8q5s1a5YCAwMVHR2thQsXqrS0tMZ7igMCAmSM0UcffSSpatE5SRoyZEid65OqXqt0vS69Lm5ubmrXrp3KysocY/b+eXl5OcaCgoIkSceOHbvuY9dXbf3s0KGDpk+frnXr1ikrK0vGGKWmpuqJJ56Q1PjXG01PZWWltm/froCAAPXv31979+7VW2+9pezsbC1fvlzdu3e3ukQAAADcxJpEGL9e9iD4448/1hjv1KmTJOn222+/7n3n5eXp+PHjunDhwmWfVVRUXPd+7b788kvdd9998vX11WuvvabWrVvX+HzatGlas2aNIiIi9OKLL2r27Nl644039OabbzqlvhvRpUsXSarxmrLGkpOTo/Ly8lr7KVUFdk9PTy1btkzp6ekaOHCg448VTbmfaFgFBQVavny5evbsqdGjR0uStm3bpsOHD2vmzJmWvVEAAAAAtxaXDuP2GXD77LFdZmamJNVY6bj6LGld+Pv7Oxb0qu7QoUNatWrV9ZRbQ3h4uMrLyzV8+HBJVbN0khzvPa+oqNB3332nAwcO6G9/+5tSUlL0+uuvO8JjXeur73k3hLy8PEn/v//2GeXS0lJJVedqv8PBVHvPe31rNcZo6tSpcnd3r7WfktSxY0dNmTJFq1ev1ooVKzR+/HjHZ419vWG9I0eOaObMmerSpYtef/11jR49Wj/88IN27dqlwMDAK96hAwAAADSW67+HuQHZbw2vHpzKy8slSefPn3fchl5SUiKpKqi6u7tr7ty5Sk5O1sqVKxUeHq4777xTkhQdHa2HH35Y06ZNkyT16tVLp0+fVkZGhmMV5MLCQsd/vb29JckREIuKijRq1Cj17NlT8+fP16lTpzR06FAdPnxYX375pZKTk2vUU1RUdMWZ2Gs5ffq0CgsLtXPnTp09e1b5+fmSqmbMu3Tpog8//FDbt2/Xfffdp+PHj8vb21sdO3aUr6+vPDw86lTfjh07FBYWpqSkJEdIvZri4mJJVavb2285t5+fMcYRVM6fP++4PtVf2XTx4kXHHwp2796t/v37a/LkyZKqgu6RI0e0YMEChYeHa8eOHY5g/sknnyggIOCK1yg7O9txzMrKyhorUxcUFGjOnDlq0aKFmjVrVms/7bP0s2fP1ooVK5SRkaHevXs79tfY1xvWKC8v15YtWxQdHa3PPvtMffv21eLFi/Xss8/WeLQCAAAAcLqGfgq9vgu4nTlzxrzwwgtGkvH09DS7du0y//jHP4y7u7uRZKZPn25yc3PNihUrHAtqRUVFmbNnzxpjqlZUnzNnjgkICDCzZs0yc+bMMW+88YYpKSlxHGPevHmmc+fOJjk52VRUVJi3337bsa+ZM2ea8+fPm6ioKMfYrFmzTElJiTlx4oQJDAw07dq1M506dTITJ040OTk5pqioyLzxxhuO7SdOnGgOHjxYrz6tWrXKeHt7mwcffNCkpaWZZcuWmbZt25qgoCCTm5trdu7caXx8fC5bTKxjx44mOTnZGGOuWp/dzp07zZ133mlSU1OvWsdnn31mXnrpJcf+x4wZY7Zu3WpWrVrlGJs/f77Jz893LMwmybz00kumuLjYsRDbO++8Y86ePWt++ukn89Zbb9VYZfzo0aNm4MCBplWrViYgIMAcPXrUPProo+aZZ54xGzZsMCUlJTWukTHGpKammqCgIMfx/Pz8zOOPP24ef/xxc8899xhPT08jyaxdu7ZO/axuxIgRZv369Zf1ojGvtzEs4OZM2dnZ5q9//avp0qWLcXd3N6NGjTK7du0ylZWVVpcGAAAAGGOMzc2YatPRDcBmsyk0NFQNvNtbijFGH374oXJzczV37lxJVXcDZGdn69NPP9WLL76onJwci6us4u/vr6NHj7rM9b5w4YIeeOABffPNN05/NjgkJERS1e8IGkd6erqWL1+ujRs3ysvLS+Hh4XrhhRfUo0cPq0sDAAAAqktqErep30zq8tzp4cOH5e/vf9XPo6Ki9Je//EW5ubmOMXd3d3Xr1k2PPvqo7rrrrgap9VYUHR2t6dOns0jXTaSkpEQ2m01LlizRf/7zHw0YMECrVq3SM888w3UGAABAk0UYb2ANMUO8b98+SdLq1as1efJkdezYUVLVrF9UVJTi4+Nv+BgNxb76eFN+jvrAgQOaNGmSiouLVVFRoSNHjlhdEhrADz/8oPfff1/vv/++ioqKFBQUpOjoaD3yyCNWlwYAAADUyqVXU79ZrVu3TtOmTVNsbKy6du2qhx9+WMHBwTp48KDi4+PVr18/q0tUUVGRXn75ZZ06dUqSNGPGDKWlpVlc1ZV5eXmpsLBQzZo1U2Jiom677TarS8J1qqys1O7duxUSEiJ/f3/FxcVp2rRpyszMlM1mI4gDAADAZfDMOOAkPDN+/QoKCrR27VotX75cJ06c0COPPKKZM2fqySefdKziDwAAALgQnhkH0HQdPHhQ7733nuLj4+Xh4aGwsDDNmDGjSdwdAgAAANwIwjiAJqWsrEwpKSmKiYnR7t275efnp8jISEVERDTZdQkAAACA+iKMA2gSsrOzFRMTo3fffVd5eXkaPny4du3apaFDh9bpLQUAAACAKyGMA7DUvn37tGLFCm3ZskUdOnTQuHHjNHXqVN19991WlwYAAAA0GsI4AKcrKCjQpk2btHLlSn333XcaMGCAYmNj9fTTT6t58+ZWlwcAAAA0OsI4AKfZv3+/Vq9eraSkJDVv3lx//OMflZCQoPvvv9/q0gAAAACnIowDaFSFhYXauHGj3n33Xf373/9W3759tWjRIhZkAwAAwC2NMA6gUaSnpysmJkaJiYm6ePGiAgMD9c4772jYsGFWlwYAAABYjjAOoMGUlJRo+/btNV5L9uqrr2rChAnq0KGD1eUBAAAATQZhHMANO3LkiNauXas1a9bowoULCgoK4rVkAAAAwDUQxgFcl9LSUm3btk0xMTFKTU1Vr169NHfuXI0fP1533HGH1eUBAAAATRphHEC9HDt2TLGxsYqNjdW5c+c0ePBgpaSkaOTIkcyCAwAAAHVEGAdQq7KyMqWkpDhmwbt06aKIiAg999xz6tatm9XlAQAAAC6nwcO4h0fVLpkhAy4XFhZmdQn1kpWVpfj4eK1atUrZ2dkaMmSINm3apCeffNLxuw4AAACg/tyMMaYhd1hSUqKPP/5YFRUVDblb4Kbw4IMPqkePHlaXcU0XL17Ujh079N5772nnzp3q3LmzIiIiNHHiRGbBAQAAgIaR1OBhHIBrOnbsmBISEvThhx/q1KlTGjJkiCZNmqTRo0erefPmVpcHAAAA3EySuM8UuIUVFxcrOTlZsbGx+vzzz3XXXXfpT3/6k8aNGydfX1+rywMAAABuWoRx4BaUnp6u9evXKz4+XgUFBRo8eDDPggMAAABOxP91A7eI/Px82Ww2rV69Wl9//bX8/Pw0d+5cjRs3Tj4+PlaXBwAAANxSCOPATayyslJ79uzR+vXrtXnzZhljNHLkSL399tsaOnQobz0AAAAALEIYB25C9leSxcTE6Pjx4xowYICWLl2qp59+Wm3atLG6PAAAAOCWRxgHbhJlZWX65JNPFBcXpy1btqhNmzYKDg7Wc889p/vvv9/q8gAAAABUQxgHXNzhw4e1bt06ffDBB8rLy9OQIUOUmJioUaNGydPT0+ryAAAAAFwBYRxwQYWFhdq6davi4uK0e/dudevWTePHj9eUKVPUvXt3q8sDAAAAUAvCOOBC0tPTFRMTo8TERJWXlysoKEi7du1iMTYAAADAxRDGgSbuf//7n+Lj4xUfH68ffvhB/fv3V1RUlMaMGaO2bdtaXR4AAACA60AYB5qgn3/+WTabTfHx8dq/f786deqksLAwjR07Vr/+9a+tLg8AAADADSKMA02EfTX0pKQkbd68WZWVlRo2bJg2bdqk0aNHq3nz5laXCAAAAKCBEMYBi6Wnp2v9+vXasGGD8vLyNGjQIC1dulRhYWHy9va2ujwAAAAAjYAwDlggIyNDGzZsUGxsrI4dO6Zf/epXmjp1qsaOHauePXtaXR4AAACARkYYB5wkPz9f27ZtU1xcnFJTU9WuXTs99dRT+uCDD/Too49aXR4AAAAAJyKMA42otLRUH3/8seLi4vTRRx/Jw8NDo0eP1kcffaTf/e53cnd3t7pEAAAAABYgjAMNrKKiQmlpaUpKSlJiYqJ+/vlnDRo0SCtXrtTTTz+tNm3aWF0iAAAAAIu5GWOM1UUArq6yslL79u3Txo0blZycrLNnz2rgwIEKCwtTSEiI7rrrLqtLBAAAANB0JDEzDtyA//73v0pKSlJcXJyOHz+uvn37aurUqRozZozuueceq8sDAAAA0EQRxoF6sgfwDRs26Pvvv1f37t01atQojR07Vv3797e6PAAAAAAugDAO1MHJkye1detWxcXFKT09XV27dtXvf/97xcbG6pFHHpGbm5vVJQIAAABwIYRx4CqysrKUnJyspKQk7d+/X+3atdOIESO0aNEiDRkyRM2aNbO6RAAAAAAuqkks4JaUlKSkpCSrywAc0tPTdeLECXl4eKhr167q1q2bfHx8asyAd+3aVUuWLLGwSgAAAAAuKqlJhPGQkBClpaVp0KBBVpcCSJKys7MlSZ07d77iDHhmZqYOHDigJvDrAwAAAMD1NJ3V1AcNGiSbzWZ1GUCd2Gw2hYaGWl0GAAAAABfFQ68AAAAAADgZYRwAAAAAACcjjAMAAAAA4GSEcQAAAAAAnIwwDgAAAACAkxHGAQAAAABwMsI4AAAAAABORhgHAAAAAMDJCOMAAAAAADgZYRwAAAAAACcjjAMAAAAA4GSEcQAAAAAAnIwwDgAAAACAkxHGAQAAAABwsps6jP/000+y2WxauHDhTXk8AAAAAIBrumnD+OHDh/Xmm28qNDRUcXFxlh7voYce0pw5cxq9Bkn6/PPPNW/ePLm5ucnNzU3h4eFKSUlxyrGv5dNPP1VwcLCjrsmTJ+uLL76wuiwAAAAAsISbMcZYXURISIgkyWazNeh+S0pK1LJlS/n5+enIkSMNuu/6HC8sLEx9+vTR/PnzHWOZmZnq1q1bo9XSvXt3ZWRk6MKFC2rVqlWjHedaLj3H4uJieXl56e6779bJkyctqamh2Gw2hYaGqgn8+gAAAABwPUk37cy4JLVo0aJJHG/jxo01gviJEyc0ZsyYRq2lZcuWkmRZEL/SOdprsdcGAAAAALcqD6sLuNWcOnVKI0eOVEVFhdWlNJpb4RwBAAAA4Ea43My4MUZpaWmaPXu2evTooTNnzugPf/iD2rdvr3vvvVebN2++6ndjYmIczyxLUmFhoRYvXuwYq6io0D//+U89//zz6tGjh7KysvR///d/uvvuu3Xu3Dl9//33euqpp/TSSy/p2Wef1W9/+1t98803Vz1eRUWFbDabxo4dq8cee0yStHbtWh06dEhnzpzRn//8Z0lSfHy8WrVqJTc3Ny1atEgXL16UJCUkJMjT01Nr166VJO3Zs0ddu3bV3r17692zlJQUTZo0SV27dtW5c+c0duxYdejQQffee6/+9a9/1bmvtfXwaudYX9fqdV379csvvygqKkoRERH6zW9+o2HDhunbb7+t9ToDAAAAQKMzTUBwcLAJDg6u07YXL14027dvNy1atDCSzLRp08zevXtNQkKCad26tZFk9u3b59hekvHz83P87Ovray49bftYSUmJ+eKLL0zLli2NJBMZGWl27dplIiIizPnz503v3r2Nr6+vMcaYsrIyc/vtt5t+/frV2Nelxzt58uRlY5f+bIwxr7zyipFkvvvuuxrfHT16tOPnrVu3mpYtW5pt27bV2ic/Pz/HeVZWVprMzEzj5eVlJJkFCxaYH3/80cTFxRlJZuDAgfXq67V6eK1zvNb4pWrrdV36NWHCBHP48GHHzwEBAcbHx8fk5ORc8zrXxaZNmy7rAQAAAADUka1JpIn6hHG7Pn36GEmmqKjIMbZ06VIjyYSGhjrGLg1/1UPq1cbuueceI8nk5eXV2G7x4sUmMTHRGGNMRUWF8fX1NR4eHjW2ufR4lZWVdQrjubm5pnXr1mbChAmOscjISLN9+/Ya25WXl1+lIzVd6Tzt51W9Nh8fH+Pp6ekYq0tf69LDGw3jtfW6tn4dOHDASLriP/s2V7vOdUEYBwAAAHADbC53m7pds2ZVpXt5eTnGgoKCJEnHjh27oX3bb7du3759jfFZs2YpMDBQ0dHRWrhwoUpLSx23SNe2r9p06NBB06dP17p165SVlSVjjFJTU/XEE0/U2M7D4/of87+0Fjc3N7Vr105lZWWOscbsa33U1uva+vXVV1+pX79+MsZc9m/kyJGSrn6dAQAAAKCxuWwYv5IuXbpIUqO9MuzLL7/UfffdJ19fX7322mtq3bp1g+5/1qxZ8vT01LJly5Senq6BAwfeUPhuKI3d1+pycnJUXl5ep15fq195eXk6fvy4Lly4cNn3WFgOAAAAgNVuqjCel5cnSRo2bNhVt7HPhpaWlkqSKisrVVBQIEm1vjM6PDxc5eXlGj58uOO7dfnelVxpRr1jx46aMmWKVq9erRUrVmj8+PF1+l5ju7Svde1hfWs1xmjq1Klyd3evU6+v1S9/f3/HAm7VHTp0SKtWrapXXQAAAADQ0Fw+jFcPfLt371b//v01efJkSVJxcbEkqaSkxLGNv7+/JGnBggU6duyYli9f7giVn3zyiSoqKhzbFxUV1TjW6dOnlZWVpZ07dyohIUH5+fmSqmbMMzMzr3i88+fPS6paddyuV69eOn36tDIyMi47n9mzZ6usrEwZGRnq3bt3jc927Nihtm3b6u9//3utfbHXUn1m2F5X9UBrr6+8vLzG96/V17r08ErnmJ2d7TimPVzbFRQUaPLkyWrRooWaNWtWa69r69eoUaPUs2dPzZ8/X+PHj1dCQoJeffVVPf/88xo3blyNflx6nQEAAACgsbl8GF+2bJlyc3OVk5Oj06dPa+/evWrevLmOHz+uefPmSZJOnjypZcuW6dy5c4qKitLAgQO1ZMkSPffccxoxYoT69eunZ555RllZWXr99dd18uRJSVW3QX/99deOY0VGRsrb21uvvvqqevXqpVdeeUVt27ZVZGSkzpw5c9nxTp06pcjISElVQX7JkiUqLCxUcHCwvL299dVXX112Pp07d1ZAQIAiIiIu++y2226Tt7e3brvttqv24/PPP9e8efMcgXXSpElKSUlRdHS047wWLlyogoICLVu2zBGQX3vtNf3yyy+19lXSNXuYn5+vixcvXnaOe/bs0ZQpUyRVhfK+fftq8ODBGjx4sPz8/OTj46M1a9YoICCg1l63atWq1n61aNFCe/bsUWBgoLZu3arZs2crJydHCQkJcnd315tvvnnV6wwAAAAAjc3NXM891g0sJCREkmSz2er8HX9/fx09evS6bhFvyi5cuKAHHnhA33zzTY3Q6Syu1ler+mWz2RQaGuoyfQIAAADQpCS5/Mz4zSY6OlrTp0+3JIi7IvoFAAAAwBVZv1T3dbI/C11UVNTgq5o724EDBzRp0iQVFxeroqJCR44csawWV+hrU+oXAAAAAFwPl5sZLyoq0ssvv6xTp05JkmbMmKG0tDSLq7oxXl5eKiwsVLNmzZSYmHjNZ8Ibiyv1tSn0CwAAAABuhMs+Mw5YiWfGAQAAANwAnhkHAAAAAMDZCOMAAAAAADgZYRwAAAAAACcjjAMAAAAA4GSEcQAAAAAAnIwwDgAAAACAkxHGAQAAAABwMsI4AAAAAABORhgHAAAAAMDJCOMAAAAAADgZYRwAAAAAACcjjAMAAAAA4GSEcQAAAAAAnMzD6gLs0tLSFBISYnUZQJ1kZmZaXQIAAAAAF9YkwnhwcLDVJQD10q1bNw0aNMjqMgAAAAC4KDdjjLG6CAAAAAAAbiFJPDMOAAAAAICTEcYBAAAAAHAywjgAAAAAAE5GGAcAAAAAwMn+H09QyMbSkgmfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.utils.plot_model(\n",
    "    model=model, to_file=\"dnn_model.png\", show_shapes=False, rankdir=\"LR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run and evaluate model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train and evaluate.\n",
    "\n",
    "We've built our Keras model using our inputs from our CSV files and the architecture we designed. Let's now run our model by training our model parameters and periodically running an evaluation to track how well we are doing on outside data as training goes on. We'll need to load both our train and eval datasets and send those to our model through the fit method. Make sure you have the right pattern, batch size, and mode when loading the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "**Lab Task #5:** Training and evaluating the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow_core/python/data/experimental/ops/readers.py:521: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow_core/python/data/experimental/ops/readers.py:215: shuffle_and_repeat (from tensorflow.python.data.experimental.ops.shuffle_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use `tf.data.Dataset.shuffle(buffer_size, seed)` followed by `tf.data.Dataset.repeat(count)`. Static tf.data optimizations will take care of using the fused implementation.\n",
      "Train for 312 steps, validate for 10 steps\n",
      "Epoch 1/5\n",
      "312/312 [==============================] - 5s 16ms/step - loss: 4.2510 - rmse: 1.5391 - mse: 4.2510 - val_loss: 1.3007 - val_rmse: 1.1402 - val_mse: 1.3007\n",
      "Epoch 2/5\n",
      "312/312 [==============================] - 3s 9ms/step - loss: 1.1924 - rmse: 1.0779 - mse: 1.1924 - val_loss: 1.2050 - val_rmse: 1.0974 - val_mse: 1.2050\n",
      "Epoch 3/5\n",
      "312/312 [==============================] - 4s 12ms/step - loss: 1.2077 - rmse: 1.0884 - mse: 1.2077 - val_loss: 1.1679 - val_rmse: 1.0804 - val_mse: 1.1679\n",
      "Epoch 4/5\n",
      "312/312 [==============================] - 4s 11ms/step - loss: 1.1964 - rmse: 1.0829 - mse: 1.1964 - val_loss: 1.2209 - val_rmse: 1.1047 - val_mse: 1.2209\n",
      "Epoch 5/5\n",
      "312/312 [==============================] - 3s 9ms/step - loss: 1.1474 - rmse: 1.0607 - mse: 1.1474 - val_loss: 1.1566 - val_rmse: 1.0752 - val_mse: 1.1566\n"
     ]
    }
   ],
   "source": [
    "# TODO 5\n",
    "TRAIN_BATCH_SIZE = 32\n",
    "NUM_TRAIN_EXAMPLES = 10000 * 5  # training dataset repeats, it'll wrap around\n",
    "NUM_EVALS = 5  # how many times to evaluate\n",
    "# Enough to get a reasonable sample, but not so much that it slows down\n",
    "NUM_EVAL_EXAMPLES = 10000\n",
    "\n",
    "# TODO -- Your code here.\n",
    "\n",
    "steps_per_epoch = NUM_TRAIN_EXAMPLES // (TRAIN_BATCH_SIZE * NUM_EVALS)\n",
    "\n",
    "logdir = os.path.join(\n",
    "    \"logs\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
    "tensorboard_callback = tf.keras.callbacks.TensorBoard(\n",
    "    log_dir=logdir, histogram_freq=1)\n",
    "\n",
    "history = model.fit(\n",
    "    trainds,\n",
    "    validation_data=evalds,\n",
    "    epochs=NUM_EVALS,\n",
    "    steps_per_epoch=steps_per_epoch,\n",
    "    callbacks=[tensorboard_callback])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize loss curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAFNCAYAAACuWnPfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8XHd97//XZxZto3VkOV5kayaJkzh2HC/SODQsAQo37GtIKLRNLpCG9ldoS5dw2wsUyqU8LqUpW2kKFEghNA0EKITesGRhjS07TuLY2SXH8ipr36WZ+f7+OCNbVmRbY2t05kjv5+MxD43mnJn5SLa/fs/3fM73mHMOEREREfFPyO8CRERERBY7BTIRERERnymQiYiIiPhMgUxERETEZwpkIiIiIj5TIBMRERHxmQKZFA0z+6qZ/d0s9203s98+19cRETlbczVmiYACmYiIiIjvFMhERESKiJmF/a5B5p8CmeQlN+3+F2b2iJkNmdmXzew8M/uRmQ2Y2U/MrG7K/q83s8fMrNfM7jOztVO2bTKznbnn/QdQNu29Xmtmu3LP/ZWZbTjLmt9jZk+bWbeZfd/MVuQeNzP7RzM7amb9Zvaoma3PbXu1me3J1XbAzP78rH5hIuKrIIxZuUOf/2xmd5vZEPDS3GNfyNU5aGa/NLNlZnaLmfWY2eNmtmnKa/xVbqwaMLMnzOzlucdDZnazmT1jZl1mdoeZxc/19ypzT4FMzsZbgFcAFwGvA34E/C+gAe/v1PsAzOwi4HbgT3Lb7gb+y8xKzKwE+C5wGxAH/jP3uuSeuwn4CvAHQD3wL8D3zaw0n0LN7GXAJ4C3AcuBfcC3cptfCbw493PU5Pbpym37MvAHzrkqYD3ws3zeV0SKShDGrN8BPg5UAb/IPfY24G+AJcAY8GtgZ+77O4FP5977YuD/A1pyY9b/ANpzr/HHwBuBlwArgB7g87OsSeaRApmcjc8654445w4APwcedM495JwbBe4CJj+1XQv80Dn3Y+fcBPApoBz4LeAKIArc4pybcM7dCWyf8h43Av/inHvQOZdxzn0Nb0C6Is9a3wF8xTm30zk3BnwQeIGZJYAJvMHvEsCcc3udc4dyz5sALjWzaudcj3NuZ57vKyLFIwhj1vecc790zmVzdQHc5ZzbMaXOUefc151zGeA/ptSdAUrxxqyoc67dOfdMbttNwF875zpyY+BHgLeaWWSWdck8USCTs3Fkyv2RGb6vzN1fgTcjBYBzLgvsB1bmth1wJ1/dft+U+03AB3JT/71m1gusyj0vH9NrGMSbBVvpnPsZ8Dm8T4tHzexWM6vO7foW4NXAPjO738xekOf7ikjxCMKYtf9s63bOPY03q/cRvLHsW5OtGbm67ppS0168AHfeLOuSeaJAJoV0EG8wALyeLbwB6gBwCFiZe2zS6in39wMfd87VTrlVOOduP8caYniHEw4AOOc+45zbAlyKdzjjL3KPb3fOvQFYineY4o4831dEgsfPMcudeZfTPNm5bzrnXpir3wGfnFLXq6bVVZabLZQiokAmhXQH8Boze7mZRYEP4E3h/wqvFyINvM/Momb2ZiA15bn/CtxkZltzzfcxM3uNmVXlWcPtwA1mtjHXy/F/8A5XtJtZS+71o8AQMApkc/0i7zCzmtxhi34gew6/BxEJhmIYs/JmZheb2ctyY9wo3uzZ5Jj1ReDjZtaU27fBzN5Q6JokfwpkUjDOuSeAdwKfBY7hNdO+zjk37pwbB94MXA904/VufGfKc1uB9+AdUuwBns7tm28NPwH+N/BtvE+4FwDX5TZX4w2iPXiHHrqA/5vb9rtAu5n14/VgvCPf9xaRYCmGMesslQJ/n6v5MN7M/gdz2/4J+D5wj5kNAL8Bts5TXZIHO/lwuIiIiIjMN82QiYiIiPhMgUxERETEZwpkIiIiIj5TIBMRERHxmQKZiIiIiM8Cd+mEJUuWuEQi4XcZIjKPduzYccw51+B3HXNBY5jI4jLb8StwgSyRSNDa2up3GSIyj8xs35n3CgaNYSKLy2zHLx2yFBEREfGZApmIiIiIzxTIRERERHwWuB6ymUxMTNDR0cHo6KjfpSwYZWVlNDY2Eo1G/S5FZMHTGDa3NH5JEC2IQNbR0UFVVRWJRAIz87ucwHPO0dXVRUdHB8lk0u9yRBY8jWFzR+OXBNWCOGQ5OjpKfX29BrI5YmbU19fr07rIPNEYNnc0fklQLYhABmggm2P6fYrML/2bmzv6XUoQLZhA5qfe3l6+8IUv5P28V7/61fT29hagIhGR2dMYJuI/BbI5cKrBLJ1On/Z5d999N7W1tYUqS0RkVjSGifhvQTT1zySbdfSNTFAWDVNeEi7oe918880888wzbNy4kWg0SllZGXV1dTz++OM8+eSTvPGNb2T//v2Mjo7y/ve/nxtvvBE4sWL34OAgr3rVq3jhC1/Ir371K1auXMn3vvc9ysvLC1q3iBSvobE0E5kstRUlBX8vjWEiRcA5F6jbli1b3HR79ux53mOZTNY90tHrDvYOP2/bXGtra3Pr1q1zzjl37733uoqKCvfss88e397V1eWcc254eNitW7fOHTt2zDnnXFNTk+vs7HRtbW0uHA67hx56yDnn3DXXXONuu+22gtd9JjP9XkX8ALS6Ihh/5uI22zFsX9eQ23Owz2Wz2fx/YXlaiGOYxi8pFrMdvxbcDNnf/tdj7DnYD8DIRAaA8ui5zZBduqKaD79u3az3T6VSJ51u/ZnPfIa77roLgP379/PUU09RX19/0nOSySQbN24EYMuWLbS3t59TzSISTJNjWDqTZSydpaIkfM5N6hrDRIrfggtkU4VDxkQmO+/vG4vFjt+/7777+MlPfsKvf/1rKioquOqqq2Y8Hbu0tPT4/XA4zMjIyLzUKiLFKRTyQljGQWSeTxrUGCYy/xZcIJv6KbB/ZIL2riHOb6iksrRwP2pVVRUDAwMzbuvr66Ouro6Kigoef/xxfvOb3xSsDhEJvskxzDnH3kMDVJVFWBWvKOh7agwT8d+CC2RTVeSa+YfH0gUNZPX19Vx55ZWsX7+e8vJyzjvvvOPbrr76ar74xS+ydu1aLr74Yq644oqC1SEiC4eZESsNMzR++jMd54LGMBH/mddvFhzNzc2utbX1pMf27t3L2rVrZ9z/ySMDRMMhkktiM26XUzvd71VkPpnZDudcs991zIV8xrBjA2Mc7BvhkmXVlES0SlE+NH5JsZjt+LXg/4XHSsIMj6UJWvAUEYmV5mb552GWTET8tfADWWmEjHOM5s64FBEJirJomLAZQ2MKZCIL3YIPZBUlXu/Y0LgCmYgEi5lRURphaEzjl8hCt+ADWUkkREk4pE+YIhJIsdIwo+kMaR+W8BGR+bPgAxlARWmE4fGM+shEJHBimuUXWRQWRSCLlYSZyGQZ1ydMEQmY8pIwIfWRiSx4iyOQ5dYgK5Y+jMrKSgAOHjzIW9/61hn3ueqqq5h+avx0t9xyC8PDw8e/f/WrX01vb+/cFSoivguZUVESLqpApjFMZO4tikBWGgkRDhnDRTSgAaxYsYI777zzrJ8/fTC7++67qa2tnYvSRKSIxEojjE5kyGSLa5ZfY5jI3FkUgczMiJVECtaDcfPNN/P5z3/++Pcf+chH+Lu/+zte/vKXs3nzZi677DK+973vPe957e3trF+/HoCRkRGuu+461q5dy5ve9KaTrgP33ve+l+bmZtatW8eHP/xhwLvY78GDB3npS1/KS1/6UgASiQTHjh0D4NOf/jTr169n/fr13HLLLcffb+3atbznPe9h3bp1vPKVr9T15kQCIFYSxlG4PjKNYSJFwDkXqNuWLVvcdHv27HneY9Md7R9xD+/vcePpzBn3zdfOnTvdi1/84uPfr1271j333HOur6/POedcZ2enu+CCC1w2m3XOOReLxZxzzrW1tbl169Y555z7h3/4B3fDDTc455x7+OGHXTgcdtu3b3fOOdfV1eWccy6dTruXvOQl7uGHH3bOOdfU1OQ6OzuPv+/k962trW79+vVucHDQDQwMuEsvvdTt3LnTtbW1uXA47B566CHnnHPXXHONu+222075c83m9yoyH4BWVwTjz1zczmYMy2Sy7pH9ve5g7/DsfmF5WohjmMYvKRazHb8W3rUsf3QzHH70eQ/HnaN8PINFQxDKc2Jw2WXwqr8/5eZNmzZx9OhRDh48SGdnJ3V1dSxbtow//dM/5YEHHiAUCnHgwAGOHDnCsmXLZnyNBx54gPe9730AbNiwgQ0bNhzfdscdd3DrrbeSTqc5dOgQe/bsOWn7dL/4xS9405veRCzmXS7qzW9+Mz//+c95/etfTzKZZOPGjQBs2bKF9vb2/H4XIlJYM4xhIeDCycWto+H8X1NjmEjRK3ggM7Mw0AoccM69dtq2UuDrwBagC7jWOddeiDpCBmaQyToKcUm4a665hjvvvJPDhw9z7bXX8o1vfIPOzk527NhBNBolkUgwOjqa9+u2tbXxqU99iu3bt1NXV8f1119/Vq8zqbS09Pj9cDis6X6RUzCzrwCvBY4659bPsP0q4HtAW+6h7zjnPlqoekIhmMg4HA7D5vz1NYaJ+Gs+ZsjeD+wFqmfY9i6gxzl3oZldB3wSuPac3u0UnwINONw5SNY51iytOqe3mMm1117Le97zHo4dO8b999/PHXfcwdKlS4lGo9x7773s27fvtM9/8YtfzDe/+U1e9rKXsXv3bh555BEA+vv7icVi1NTUcOTIEX70ox9x1VVXAVBVVcXAwABLliw56bVe9KIXcf3113PzzTfjnOOuu+7itttum/OfWWSB+yrwObwPjafy8+kfNM/ZKcawsZEJ2ruGOH9JjMqy6Jy+JWgME/FbQQOZmTUCrwE+DvzZDLu8AfhI7v6dwOfMzHLHXOdcrCRC58AYmawjHJrbT5jr1q1jYGCAlStXsnz5ct7xjnfwute9jssuu4zm5mYuueSS0z7/ve99LzfccANr165l7dq1bNmyBYDLL7+cTZs2cckll7Bq1SquvPLK48+58cYbufrqq1mxYgX33nvv8cc3b97M9ddfTyqVAuDd7343mzZt0tS+SB6ccw+YWcLvOiZNXmh8aDxTkECmMUzEX1ag7OO9uNmdwCeAKuDPZzhkuRu42jnXkfv+GWCrc+7YtP1uBG4EWL169Zbpn9T27t3L2rVrz1jPwOgEbceGSC6JUVWAAW2hme3vVaTQzGyHc67Zh/dNAD84zSHLbwMdwEG8Me6xM71mc3Ozm74+12z/rT11ZIBwyDi/oXI25S9qGr+kWMx2/CrYshdmNtl7seNcX8s5d6tzrtk519zQ0HDWr1NREsbQJUhEZE7sBJqcc5cDnwW+e6odzexGM2s1s9bOzs6zfsNY7jJwWV0GTmTBKeQ6ZFcCrzezduBbwMvM7N+n7XMAWAVgZhGgBq+5vyDCoRBl0XDRLRArIsHjnOt3zg3m7t8NRM1sySn2nZMPlbHSMFnnGNGHSpEFp2CBzDn3Qedco3MuAVwH/Mw5985pu30f+P3c/bfm9inoRz99whSRuWBmy8zMcvdTeONpwT5QwtQLjetDpchCM+/rkJnZR/EWSfs+8GXgNjN7GujGC25nxTlHbmw8rVhJmGODjtHxDBWlC28ZtrlS4FwsUvTM7HbgKmCJmXUAHwaiAM65L+J9iHyvmaWBEeC6c/lAOZsxLBIOURoJMzyW8TpzZUYavySI5iWROOfuA+7L3f/QlMdHgWvO9fXLysro6uqivr7+jAPaZAgbUiA7JeccXV1dlJWV+V2KiG+cc28/w/bP4S2Lcc7yGcNipWH6RiZm/SF0sdH4JUG1IBJJY2MjHR0dzLZZtqtvlP5DxrHK0jPvvEiVlZXR2Njodxkii0I+Y9jweJruoQky3aVEw4vicsR50/glQbQgAlk0GiWZTM56/6/858P8ZO8RdvzNKwjN8XpkIiL5ymcMO9g7wlv+/md8+HWXcsOVsx/3RKS4LcqPVy3JOD3DEzx7bNDvUkRE8rKitpzGunK2tXX7XYqIzKHFGcgScQC2tfX4XImISP5SyTjb2rrVvC6ygCzKQJaor2BJZSmt7fqEKSLBszUZp2tonGc6h/wuRUTmyKIMZGZGS6KObQpkIhJAqWQ9gA5biiwgizKQgXfYsqNnhEN9I36XIiKSl0R9BQ1VpWxrK+g6tCIyjxZtIEslvT6y7e3qIxORYDEzUsk4D6qPTGTBWLSB7JJlVcRKwmzXlL+IBNDWZJxDfaN09GiWX2QhWLSBLBIOsbmpju3qIxORAJqc5VcfmcjCsGgDGXh9ZE8cGaBvZMLvUkRE8nLR0ipqyqMKZCILxKIPZM7Bjn0a0EQkWEIhoyUR19niIgvEog5kG1fVEg2bGvtFJJC2JuO0HRviaP+o36WIyDla1IGsvCTM+pU1auwXkUA63kemWTKRwFvUgQwglYjzSEcfoxMZv0sREcnLuhXVVJSE1UcmsgAs+kDWnIgznsnySEef36WIiOQlEg6xpalOgUxkAVAga6oD0PIXIhJIW5Pe2eK9w+N+lyIi52DRB7K6WAkXnVepQCYigTR5tnirTk4SCbRFH8jAO2y5o72HTFaXIBGRYLl8VS0l4ZAa+0UCToEMr7F/YCzNE4cH/C5FRCQvZdEwG1fV8qD6yEQCTYEMaE6oj0xEgiuVjLP7QB9DY2m/SxGRs6RABjTWVbCipkxT/iISSKlknEzWsfM59ZGJBJUCWU5LMk5rezfOqY9MRIJlc1Md4ZBp+QuRAFMgy2lOxDnSP8b+7hG/SxERyUtlaYT1K6rVRyYSYApkOamEdwkS9ZGJSBClknF27e/VVUdEAkqBLGfN0kpqyqMKZCISSKlkPeNpXXVEJKgUyHJCIaO5qU6N/SISSC25s8W3tXX5XImInA0FsilaknGe7Ryia3DM71JERPJSW1HCJcuq1EcmElAKZFO0HF+PTKeOi0jwpJJxduzrIZ3J+l2KiORJgWyKy1bWUhoJqY9MRAIplYwzPJ7hsYP9fpciInkqWCAzszIz22ZmD5vZY2b2tzPsc72ZdZrZrtzt3YWqZzZKIiE2rqqlVYFMRAJo8mxxrUcmEjyFnCEbA17mnLsc2AhcbWZXzLDffzjnNuZuXypgPbPSkoiz+2C/LkEiIoGztLqM5JKY+shEAqhggcx5BnPfRnO3ol8GvyV3CZJd+3v9LkVEJG+pRJzt7d1ks0U/3IrIFAXtITOzsJntAo4CP3bOPTjDbm8xs0fM7E4zW1XIemZj8+paQqYpfxEJplQyTt/IBE8dHTzzziJSNAoayJxzGefcRqARSJnZ+mm7/BeQcM5tAH4MfG2m1zGzG82s1cxaOzs7C1kyVWVR1i6vVmO/iARSKjnZR6b1yESCZF7OsnTO9QL3AldPe7zLOTe56NeXgC2neP6tzrlm51xzQ0NDYYvF6yN76LleJnTquIgETGNdOStqytRHJhIwhTzLssHManP3y4FXAI9P22f5lG9fD+wtVD35aEnEGZnQqeMiEjxmRksyzra2bpxTH5lIUBRyhmw5cK+ZPQJsx+sh+4GZfdTMXp/b5325JTEeBt4HXF/AemZtcoFYLX8hIkGUSsY5OjDGvq5hv0sRkVmKFOqFnXOPAJtmePxDU+5/EPhgoWo4W0ury2iqr2BbWzfvftH5fpcjIpKXrckT65EllsR8rkZEZkMr9Z9CSyJO674eTfmLSOBc0FBJPFaiPjKRAFEgO4VUIk730DjPdA75XYqISF7MjFQizrZ2nWkpEhQKZKfQfPxC4/qEKSLBk0rG2d89wsHeEb9LEZFZUCA7heSSGEsqSxTIRBYpM/uKmR01s91n2K/FzNJm9tb5qm02Jtcj0xgmEgwKZKdgZjQ3xTWYiSxeX2Xa2onTmVkY+CRwz3wUlI+1y6upKo2oj0wkIBTITqMlN+V/uG/U71JEZJ455x4AzpRm/hj4Nt7l4YpKOGQ0J+p0GTiRgFAgO41UQlP+IjIzM1sJvAn4Z79rOZVUsp6njw5ybHDszDuLiK8UyE5j7fIqYiVhBTIRmcktwF855854jbX5vB7vVJN9ZFrkWqT4KZCdRiQcYnNTHdvbe/wuRUSKTzPwLTNrB94KfMHM3jjTjvN9Pd5Jl62soSwaUh+ZSAAokJ1Bc1Ocxw/30zcy4XcpIlJEnHNJ51zCOZcA7gT+0Dn3XZ/LOklJJMTm1eojEwkCBbIzaEnW4Rzs3KdZMpHFxMxuB34NXGxmHWb2LjO7ycxu8ru2fKSScfYc6qd/VB8qRYpZwa5luVBsWlVHJGRsb+/mpZcs9bscEZknzrm357Hv9QUs5ZykknGcgx3tPRrDRIqYZsjOoLwkzPqVNWrsF5FA2rSqjmjY1EcmUuQUyGYhlYzz8P4+RicyfpciIpKX8pIwGxpr9aFSpMgpkM1Cc1Md45ksjx7o87sUEZG8pZJxHunoZWRcHypFipUC2Sy05BaI1ZlKIhJEqUSciYzjof06OUmkWCmQzUJdrIQ1Syu1uKKIBNKWRB1m+lApUswUyGapORGndV8PmazzuxQRkbxUl0W5dHm1AplIEVMgm6VUso6B0TRPHhnwuxQRkbylknF2PtfDePqMV3oSER8okM1Sc5MuNC4iwbU1GWd0QicniRQrBbJZaqwrZ3lNmab8RSSQdHKSSHFTIJslM6MlEWd7ezfOqY9MRIKlvrKUC5dWsq2ty+9SRGQGCmR5aEnUcaR/jI6eEb9LERHJWyoZp7VdJyeJFCMFsjy0JNVHJiLBtTUZZ2Aszd5D/X6XIiLTKJDl4aKlVVSXRRTIRCSQ1EcmUrwUyPIQChnNibgGMxEJpBW15ayKl2sMEylCCmR5aknEeaZziK7BMb9LERHJWypRzzadnCRSdBTI8tSSqAOgdZ+uCSciwbM1Gad7aJxnOgf9LkVEplAgy9NljTWUREK6rqWIBFIqd3LSgzpsKVJUFMjyVBoJs7Gxlm3tmiETkeBpqq9gaVWp+shEiowC2VloSdbx2IE+hsfTfpciIpIXMyOVjPPgs+ojEykmBQtkZlZmZtvM7GEze8zM/naGfUrN7D/M7Gkze9DMEoWqZy61JOKks45dz/X6XYqISN62JuMc7h/VItciRaSQM2RjwMucc5cDG4GrzeyKafu8C+hxzl0I/CPwyQLWM2c2N9VhBtvURyYiAZRK1gNaj0ykmBQskDnP5Gk80dxt+vz4G4Cv5e7fCbzczKxQNc2V6rIoa5dVa4FYEQmkNUsrqa2IKpCJFJGC9pCZWdjMdgFHgR875x6ctstKYD+Acy4N9AH1haxprrQk6njouV4mMlm/SxERyUsoZDQ3xTXLL1JEChrInHMZ59xGoBFImdn6s3kdM7vRzFrNrLWzs3NuizxLLck4w+MZ9hzUNeFEJHi2JuO0HRviaP+o36WICPN0lqVzrhe4F7h62qYDwCoAM4sANUDXDM+/1TnX7JxrbmhoKHS5szJ5TTgdthSRIJpcj0yzZCLFoZBnWTaYWW3ufjnwCuDxabt9H/j93P23Aj9zATkP+7zqMlbHKxTIRCSQ1q2opqIkrD4ykSIRKeBrLwe+ZmZhvOB3h3PuB2b2UaDVOfd94MvAbWb2NNANXFfAeuZcSyLOfU8cxTlHAM5FEBE5LhIOsaWpToFMpEgULJA55x4BNs3w+Iem3B8FrilUDYXWkqjj2zs7ePbYEBc0VPpdjohIXrYm43zqnifpHR6ntqLE73JEFjWt1H8OWnI9GNv1CVNEAmhyPbLtuhSciO8UyM7B+Uti1MdKNJiJSCBtaKyhJBJiW9vzzqUSkXmmQHYOzIzmRJ0a+0UkkMqiYTauqlUfmUgRUCA7Ry2JOM91D3NEa/mISABtTcbZfbCfwbG036WILGoKZOdI65GJSJClknEyWcfOfWq9EPGTAtk5mlzLR439IhJEm1fXEQ6ZDluK+EyB7BxFwiE2r65TY7+IBFKsNML6lTUKZCI+UyCbA82JOvYe7qd/dMLvUkRE8rY1GWfX/l5GJzJ+lyKyaCmQzYFUIo5zsEM9GCISQKlEnPFMlof39/pdisiipUA2BzauriUSMlrV2C9SdMzzTjP7UO771WaW8ruuYtKSiGOGDluK+EiBbA5UlERYt7KG7W2aIRMpQl8AXgC8Pff9APB5/8opPjUVUS4+r4pt+lAp4hsFsjmSStSxq6OXsbR6MESKzFbn3B8BowDOuR5AF26cZmsyzo59PaQzWb9LEVmUZhXIzOz9Zladm/r/spntNLNXFrq4IGlOxBlPZ3m0o8/vUkTkZBNmFgYcgJk1AEod07Qk4wyPZ3jsYL/fpYgsSrOdIfufzrl+4JVAHfC7wN8XrKoAam6qA9CUv0jx+QxwF7DUzD4O/AL4P/6WVHxSuUWu1Ucm4o/ZBjLLfX01cJtz7rEpjwlQX1nKBQ0xWrUemUhRcc59A/hL4BPAIeCNzrn/9Leq4rO0uozkkhgPKpCJ+GK2gWyHmd2DF8j+n5lVoSn/50kl47S2d5PNOr9LEZEcM7sAaHPOfR7YDbzCzGpn8byvmNlRM9t9iu1vMLNHzGyXmbWa2QvnuPR5l0rE2a4xTMQXsw1k7wJuBlqcc8NAFLihYFUFVEsiTv9omiePDvhdioic8G0gY2YXAv8CrAK+OYvnfRW4+jTbfwpc7pzbCPxP4EvnWKfvUsk4fSMTGsNEfDDbQPYC4AnnXK+ZvRP4G0Dd69Mcv9C4pvxFiknWOZcG3gx8zjn3F8DyMz3JOfcAcMp/zM65Qefc5FRSjNxJA0GWSqqPTMQvsw1k/wwMm9nlwAeAZ4CvF6yqgGqsK2dZdRnb1EcmUkwmzOztwO8BP8g9Fp2LFzazN5nZ48AP8WbJAq2xrpwVNWXqIxPxwWwDWTr3SfANeJ8wPw9UFa6sYDIzmhN1bG/r5sQHZxHx2Q14s/wfd861mVkSuG0uXtg5d5dz7hLgjcDHTrWfmd2Y6zNr7ezsnIu3LggzI5WMs01jmMi8m20gGzCzD+Itd/FDMwsxR58wF5pUMs7h/lE6ekb8LkVEAOfcHufc+5xzt+e+b3POfXKO3+MB4HwzW3KK7bc655qdc80NDQ1z+dZzLpWsp3OycMETAAAgAElEQVRgjPauYb9LEVlUZhvIrgXG8NYjOww0Av+3YFUF2GQfWes+TfmLFAMze62ZPWRm3WbWb2YDZnbOq5+a2YVmZrn7m4FSoOtcX9dvJ/rIAv+jiATKrAJZLoR9A6gxs9cCo8459ZDN4KLzqqgqi7BN17UUKRa3AL8P1Dvnqp1zVc656jM9ycxuB34NXGxmHWb2LjO7ycxuyu3yFmC3me3CuzbmtW4BHOe7oCFGfaxEfWQi8ywym53M7G14M2L34S0I+1kz+wvn3J0FrC2QwiGjuamO7VqxX6RY7Ad25xuWnHNvP8P2TwJzeuizGEztIxOR+TOrQAb8Nd4aZEfh+LXgfgIokM2gORHn3ieeoHtonHhM1zAW8dlfAneb2f14rRcAOOc+7V9JxS2VjPOj3Yc50DvCytpyv8sRWRRm20MWmgxjOV15PHfRmezBaNUsmUgx+DgwDJThnR0+eZNTmBzDtKaiyPyZ7QzZf5vZ/wNuz31/LXB3YUoKvg2NNZREQrTu6+GV65b5XY7IYrfCObfe7yKC5JJl1VSVRXiwrZs3blrpdzkii8KsAplz7i/M7C3AlbmHbnXO3VW4soKtNBLm8sYa9WCIFIe7zeyVzrl7/C4kKMIhoyUR15mWIvNo1ocdnXPfds79We6mMHYGLYk4uw/0MTye9rsUkUUrtyzFn+PN8o/M5bIXC10qGeeZziGODY6deWcROWenDWSTA9cMNw1oZ9CSiJPOOnbt7/W7FJFFK3dm5R7nXMg5V57PsheLnfrIRObXaQPZ5MA1w00D2hlsbqrDDLZrPTIRv+0wsxa/iwia9StqKI+G2aaTk0TmRcHOlDSzVWZ2r5ntMbPHzOz9M+xzlZn1mdmu3O1DhapnvtWUR7lkWbVW7Bfx31bg12b2jJk9YmaPmtkjfhdV7EoiITatrlUvrMg8me1ZlmcjDXzAObfTzKrwPqX+2Dm3Z9p+P3fOvbaAdfimJVHHt3d0kM5kiYS1SoiIT/6H3wUEVSoZ559++hT9oxNUl+nyxSKFVLCU4Jw75Jzbmbs/AOwFFtX50y2JOEPjGfYcUrudiF+cc/tmuvldVxCkknGcgx3tar0QKbR5mbYxswSwCXhwhs0vMLOHzexHZrZuPuqZL5MXGt+uwUxEAmjTqjqiYdN1LUXmQcEDmZlVAt8G/sQ5N32qaCfQ5Jy7HPgs8N1TvMaNZtZqZq2dnZ2FLXgOLaspY1W8XGcpiUgglZeE2dBYq/XIROZBQQOZmUXxwtg3nHPfmb7dOdfvnBvM3b8biJrZkhn2u9U51+yca25oaChkyXOuJRFne3s3eV7XWESkKKSScR7p6GNkPON3KSILWiHPsjTgy8DeU13E18yW5fbDzFK5ehbUR7GWRJyuoXHajg35XYqISN5SSW9NxYeeU+uFSCEV8izLK4HfBR41s125x/4XsBrAOfdF4K3Ae80sDYwA17kFNpV0oo+sm/MbKn2uRkQkP1ua6ggZPNjWzW9d+LwDGCIyRwoWyJxzvwDsDPt8DvhcoWooBhc0xIjHStje3sO1Lav9LkdEJC/VZVEuXVGt9chECkyLYxWYmdHcVMd2rXYtIgGVStSz87kextNZv0sRWbAUyOZBKhlnX9cwR/tH/S5FRCRvqWScsXSWRw/o2rwihaJANg+atR6ZiARYS6IOQOuRiRSQAtk8WLeimvJoWIctRSSQ6itLWbO0Un1kIgWkQDYPomHvIr0KZCISVKlknNb2HjLZBXUivEjRUCCbJy2JOHsP9TMwOuF3KSIieUsl4wyOpdmra/OKFIQC2TxJJeNkHezYpz4yEQmeVNLrhVUfmUhhKJDNk42ragmHjFY19otIAC2vKWd1vELXtRQpEAWyeRIrjbB+RTXb1EcmIgGVSsbZ3t6ja/OKFIAC2TxqTsR5eH8vY2ldpFdEgieVjNM9NM4znYN+lyKy4CiQzaOWhLe44u4DfX6XIiKSt1RCfWQihaJANo8mF1fc1qY+MhEJnqb6CpZWlWo9MpECUCCbR/WVpZzfEKNVfWQiEkBmRioZ58Fnu9VHJjLHFMjmWSoRp3VfD1ktrigiAbQ1Gedw/ygdPSN+lyKyoCiQzbPmRJy+kQmeOqqmWBEJnlSyHlAfmchcUyCbZ5NNsVr+QkSCaM3SSmorolqPTGSOKZDNs1Xxcs6rLmW7Pl2KSACFQkZLIq7GfpE5pkA2z8yM5kRcjf0iElhbk3Hau4Y50j/qdykiC4YCmQ9SiTgH+0bp6Bn2uxQRkbxNXtdSs2Qic0eBzAfNufXIdF1LEQmiS5dXEysJK5CJzCEFMh9csqyaqtKIGvtFJJAi4RBb1EcmMqcUyHwQDhlbEnVq7BeRwNqajPPEkQF6hsb9LkVkQVAg80lLIs5TRwc1mIlIIE32kW3XTL/InFAg80lLbj2y1n3qIxOR4NnQWENJJKTDliJzRIHMJxsaaygJh7T8hYgEUmkkzKZVteqFFZkjCmQ+KYuG2dBYo8FMRAJrazLO7gN9DI6l/S5FJPAUyHzUkozzaEcfI+MZv0sREclbKllP1sEOtV6InDMFMh+1JOpIZx279vf6XYqITGNmXzGzo2a2+xTb32Fmj5jZo2b2KzO7fL5r9NvmploiIdN1LUXmgAKZj7asjmOms5REitRXgatPs70NeIlz7jLgY8Ct81FUMakoibBuZQ3b2zRDJnKuFMh8VFMR5eLzqhTIRIqQc+4B4JT/OJ1zv3LOTSaR3wCN81JYkdmajLNrfy+jE2q9EDkXCmQ+a0nE2bmvh3Qm63cpInL23gX86FQbzexGM2s1s9bOzs55LKvwUok445ksD6v1QuScFCyQmdkqM7vXzPaY2WNm9v4Z9jEz+4yZPZ3rxdhcqHqKVUsyztB4hr2HBvwuRUTOgpm9FC+Q/dWp9nHO3eqca3bONTc0NMxfcfOgJeG1Xmg9MpFzU8gZsjTwAefcpcAVwB+Z2aXT9nkVsCZ3uxH45wLWU5Rachca12FLkeAxsw3Al4A3OOcWZWf7ZOuFlvAROTcFC2TOuUPOuZ25+wPAXmDltN3eAHzdeX4D1JrZ8kLVVIyW15TTWFeuQCYSMGa2GvgO8LvOuSf9rsdPW5NxduzrYUKtFyJnbV56yMwsAWwCHpy2aSWwf8r3HTw/tC14qUSc7e09OOf8LkVEcszsduDXwMVm1mFm7zKzm8zsptwuHwLqgS+Y2S4za/WtWJ+lkvUMj2d47GC/36WIBFak0G9gZpXAt4E/cc6d1b9WM7sR75Amq1evnsPqikNzIs53HjpAe9cwySUxv8sREcA59/YzbH838O55KqeotSS91ottbV1sXFXrczUiwVTQGTIzi+KFsW84574zwy4HgFVTvm/MPXaShdwQC5DKDWbb1RQrIgG0tKqM85fE1Ngvcg4KeZalAV8G9jrnPn2K3b4P/F7ubMsrgD7n3KFC1VSsLmiopK4iqj4yEQmsVDLOtrZuslm1XoicjULOkF0J/C7wslx/xS4ze/W0Hoy7gWeBp4F/Bf6wgPUULTOjORFXIBORwEol4/SPpnniiJbwETkbBeshc879ArAz7OOAPypUDUGSSsT58Z4jHB0YZWlVmd/liIjkJZWMA956ZGuXV/tcjUjwaKX+ItGcW4+stV3XhBOR4Gmsq2Blbbn6yETOkgJZkVi/soayaEiDmYgEVioZ58G2bi3hI3IWFMiKRDQcYtOqOlr3KZCJSDClknGODY7RdmzI71JEAkeBrIi0JOPsOdjPwOiE36WIiORtah+ZiORHgayIpBJxsg52PtfrdykiInk7f0mMJZUlCmQiZ0GBrIhsWl1LOGS0avkLEQkgMzveRyYi+VEgKyKx0gjrVlTr06WIBFYqEedA7wgHekf8LkUkUBTIikxzU5xd+3sZT2f9LkVEJG8tuT4yXQpOJD8KZEUmlaxjLJ3l0QN9fpciIpK3S5ZVU1UW0WFLkTwpkBWZ5kTu06X6yEQkgMIhoyURZ1tbl9+liASKAlmRWVJZyvlLYmrsF5HASiXjPNM5xLHBMb9LEQkMBbIi1JKIs729h2xWq12LSPCk1EcmkjcFsiLUnKijb2SCpzsH/S5FRCRv61fUUB4Nq49MJA8KZEVIq12LSJCVREJsbqrVGCaSBwWyIrQ6XsHSqlI19otIYKUS9ew93E/fiC4FJzIbCmRFyMw7S6m1vcfvUkREzkoqGcc52LFPHyxFZkOBrEi1JOq02rWIBNam1bVEw6Y+MpFZUiArUpPrkWn5CxEJorJomMsb1UcmMlsKZEVq7fJqqkojGsxEJLBSyTiPdvQxPJ72uxSRoqdAVqTCIWNzU50a+0UksFLJOOms46Hnev0uRaToKZAVsZZEHU8eGaR3eNzvUkRkvj33G9j3K3DBXSB6S1MdIUN9ZCKzoEBWxFqO95HpbEuRRefn/wD/9ir47Gbvfv9BvyvKW1VZlHUranRdS5FZUCArYpev8s5S2q7TxkUWn2u+Cm/6F6haAT/9KPzjOvjGNbDne5AOzqx5Khnnoed6GUtn/C5FpKgpkBWxsmiYDY21uh6cyGJUEoPLr4Mbfgh/vBNe+GdweDfc8Xvw6Uvgvz8IRx7zu8ozSiXjjKWzPNrR53cpIkVNgazItSTiPHqgj9EJfboUWbTqL4CX/2/4093wjm9D4kWw7V/hn38Lbr0Ktn8JRoqzcX6y9UJ9ZCKnp0BW5FoSdUxkHLv2F+dgKyLzKBSGNb8Nb/safOAJuPqTkJmAH34A/uFi+Pa74dn7IZv1u9Lj4rES1iyt1BI+ImegQFbkmpu8T5c6bCkiJ4nVwxU3wU2/gBvvh02/C0/dA19/PXzmcrjvk9D7nN9VAt5hyx37eshkg3vGqEihKZAVuZqKKBefV8X2fTrTUkRmYAYrNsJrPgUfeBLe8mWIXwD3fQJu2QBffyM8eidMjPpWYioZZ3Aszd5D/b7VIFLsIn4XIGfWkqzjuw8dJJN1hEPmdzkiUqyiZXDZW71b73Ow63bY9e/w7XdBWQ1c9jbY9E5YfrkX5OZJKnmij2z9ypp5e1+RINEMWQC0JPTpUkTyVLsarvoreN/D8HvfgzWvhJ1fh1tfAl98EfzmizA8P60Qy2vKWR2v0HpkIqehQBYAk2cp6TJKIpK3UAjOvwre8iX48yfgNf8A4Qj89195JwLc8fvw1E8gW9gzuVPJONvaunEBvvKASCEVLJCZ2VfM7KiZ7T7F9qvMrM/MduVuHypULUG3oraclbXlCmQicm7K66Dl3XDjfXDTL737bQ/AN94Ct1wGP/0YdD9bkLdOJeP0DE/w9NHBgry+SNAVcobsq8DVZ9jn5865jbnbRwtYS+C1JOrY3t6jT5ciMjeWrYerP+Etn/G2r8N56+AXn4bPbIJ/e43XfzY+NGdvtzWp9chETqdggcw59wCgf3lzpCUZp3NgjH1dw36XIiILSaQELn0DvOM/4U8fg5d/CAYOwndvgk9dDP/1fuhoPeeLnK+OV3BedanWIxM5Bb97yF5gZg+b2Y/MbN2pdjKzG82s1cxaOzs757O+opHK9ZFt02FLESmU6hXwog94l2q64Udw6evhkTvgSy+HL1wBv/osDB49q5c2M1LJevWRiZyCn4FsJ9DknLsc+Czw3VPt6Jy71TnX7JxrbmhomLcCi8kFDZXUVkRpVSATkUIzg6bfgjd+Af78SXjdZ6C0Gu75G/j0WvjWO+CJH0EmndfLppJxDvePsr97pECFiwSXb+uQOef6p9y/28y+YGZLnHPH/KqpmIVCRnNTnO3tWiBWROZRaRVs+X3v1vkEPPTv8PC34PEfQOV53gXQN74TGi4640ud6CPrYnV9RaErFwkU32bIzGyZmbcyoZmlcrVokZrTaEnU0XZsiM6BMb9LEZHFqOFieOXH4M/2wHW3w8pm+NXn4PMt8OXcOmdjA6d8+oUNldRVRNVHJjKDQi57cTvwa+BiM+sws3eZ2U1mdlNul7cCu83sYeAzwHVOjQWn1ZL7dKnDliKFN4uley4xs1+b2ZiZ/fl81+ercBQueTW8/ZvwZ3vhFR+DkR74/h/Dpy6C7/4h7PvV804ECIWMlkRcvbALlXPnfPLHYlawQ5bOubefYfvngM8V6v0XovUraiiLhtjW3s2rLlvudzkiC91X8caor59iezfwPuCN81VQUao6D658H/zWH3tnYz50G+z+Duz6BsTP9y7VdPnbvRMG8PrI7tlzhMN9oyyrKfO5eDknw91wYCd0bPduB3aAhSD5Im8x4uRLvL8D83iZriDTtSwDpCQSYuOqWlrVRyZScM65B8wscZrtR4GjZvaaeSuqmJnBqhbvdvUnYM/3vX6zn34UfvZ3cOFvw6Z3csXq3wK8M8Zff/kKn4uWWctMwJHH4ECrF7w7tkPX0942C0HDWm/5lMwEtN0Pe77nbatZDee/JBfQXgyVS/36CYqeAlnApBJxPnfv0wyOpaks1R+fiBShkhhsfLt363rGmy3b9U244/dYV1HPR0u3sm/PBFy+uCcXi1r/wdzMVy6AHXwI0rmzY2MN0JiCjb8DjS2wYpN38sck57w/92fvhWfvg73f92ZOAZau88LZ+Vd5Z/KWVs7rj1XM9D96wDQn4mQd7NzXw4svWpxLgIgEjZndCNwIsHr1ap+rmWf1F3iLzb70r+GZn2EP3cbv7PkhkSfuhls3eYc0Ey+CuqS3SK3Mv/FhOPTwyYce+w9428IlsPxyaL4BVm7xAljt6tMfhjSDJRd6t9R7vOukHtoFz97vBbTtX4LffB5CEe/1zr/KO7zZ2Oz1Jy5SCmQBs7mpjpB5jf0KZCLB4Jy7FbgVoLm5eXF2PYfCsOYVsOYVfO2e7XTc/zX+enwHkR9+wNtuYahrgvo1sGQN1F+Y+7rGO8ylPqS5MTl7daD1RAA78hhkc2vK1TbB6hd4QamxxbvEVqT03N4zFPbC3Mot8KI/g4kR2P+gF86evQ/u+3u47xNQUunNmp1/lXdbeumi+nNXIAuYytII61bU6CwlEQmsDRddwMd+9ipecNVf88qGbji8G7qegmNPeX1JbfdDevTEE0qrTw5oSy70vtZfANFy/36QIBjp8Wa8OnbkZr9avccASqpg5Sa48v1e+FrZDJXz8EE/Wn4idIF3ckD7L7w/92fvg6fu8R6PNXgzZ+df5fWh1S7s2WUFsgBqTtRx+7bnGE9nKYn4ffUrkYUpt3TPVcASM+sAPgxEAZxzXzSzZUArUA1kzexPgEunLnotM9vQWENJJMS29h5euX6dd2HzqbJZ6O84EdCOPeUFtvZfwiP/MWVHg5pVJwLa1Jm16pWLanYF8K6ccHTPid6vA61w7MncRoOla+GS156Y/Wq42Ju98ltF3LtM16Wv977v6zhxeLPtfth9p/d4/PwTAS35Yu95C4gCWQClEnH+7Zft7D7Yx+bVdX6XI7IgzWLpnsNA4zyVs6CURsJsWlV76pn+UMibDaldDRe+/ORt40PeIbeup+DY0ydm1vZ/A8YHT+wXrfBm0I4HtcmZtQtPbkAPsv5DUw495hrvJ4a9bRVLvNC14VqvN2vFZiir9rfe2apphE3v8G7OQefjucOb98Ojd8KOfwMMlm84MdO26gooCfbVHxTIAqg5d6Hx7W3dCmQiEkhbk2d5xnhJzPuPePmGkx93DgYOn3zo89hTcHAn7PkuuOyJfauWTzsEmptZq119TjNGzjkGx9IcHRijc+ptcIy+kQnKo2FiJWFipREqSiNUloaJlUSIleZuuW3eY2Ei4SlHQCZGco33UwJYf4e3LRT1fh+bfy83+9Xs9YIthBlCy83sLV0LV7zXW1bjwM4Thzd//QX45T95Jx+s2noioC3fCOFgRZxgVSsANFSVklwSY3t7D3/wEr+rERHJXypZT/ZnT7NjXw8vmYsTlMygerl3S7745G3pMeh+9sShz8mZtd3fgdHeE/uFS73DYicdAl3DeO0FHMuUzxC0Rk8KXZ0DY4xOZJkuGjZqyqOMjGcYGs/M4odxNNkRUpFnaI48wwZ7mjXZNiJ4zz0WWcb+inUcXvlmjtVuYCh+KWVlFV7II0LF0TCVfT1UlESoLPXCXaw0QmkkhAU9pIWjsHqrd3vJX3ozpvt+7S2x0XY//Oxj3q20BhIvPNF/tuSiog+oCzuQ/eRvvdWh6xLerXb1uZ8tUiRaEnXcs+cI2awjFCruv2QiItNtbqolEjK2tXXNTSA7nUjpiVmWnGzW0Ts8TnfnIYYPPs7E0ScIdz9N+UAbtc/son7v3ccDUAlQ4qoZd8vpyi6n3S3nWbeCrrLVjFWuJl4do7kpRkNVKQ2Vpd7XqlKW5r7WlEePB6Fs1jEykWFoPM3QWIahsTSjA92EDz9E+ZGdVB/bRbz3UcomvKA4ZuU8V3YJ95S8jcfDF/EIazgwUc3weIbBg2mG2tKks+2z+jWEQ0ZFSZjK0sjxr7HSSC64hXOzdpEp+3hhrvL4PhEqSk88P1YS8f//n5IYrPlt7wYwdCw3e5abQXvih97jVctPLK9x/kuOXzmimCzcQDbaD7/5wsln6mDTAlrTift1iUCdWt2ciHNHawfPdA6y5rwF0g8hIotGRUmE9Str5vxC48Pj6ZNmsY5OO3Q4ef/Y4Bjp7NQVSC4ELqQsGmJpVRnLa8NcUtbNRZHDNLmDLE93sGZ0H5sGdhMdvc97ShYYjEA0ARVrIHIhxNZAPHcoNFb5vP9TQi5DrGcvsY4pK94fexJwgHmN9utfe/ysx9Kla1kTCrMGePUMP69zjvFM9ni4mxr0hsfTDJ70+NRtGQbHvMd6hkem7J+ecZbvVMqj3uxbZWmYipIIy2vKWL+yhstW1nBZYw3nVc/z5bFiS2D9W7wbQHfbyWdvPny79/iSi09cQSDxQiirmd86Z7BwA1lZNfyvQzB0FHrap932wTP3wsDBk58TKc+Fs2lBbTK8FVHDYCrXR7atvVuBTEQCaWvSO0FpdCJDWfTUvVvpTJauofHn9WUd7R89KWR1DozNeEgwZLBkyszV2uVVU2azyo4/3lBVSqwkfObDeiO9J5/9Odmz9szPIDN2Yr+ymhOHPsvjcPgRr/9pYsjbXlHvLTVx2TVe39fKzXkHAzOjNBKmNBImHpubhXUzWfe8ADcZ9CZD28nbcl/H0jzXPcy9TxxlMus2VJVy2coa1q+sYf2Kai5rrGFZddn8HTqNJ73bluu9s3eP7D4R0B76d9h2q3fppxWbTxzeXLXVl6Np5gJ2Zfbm5mbX2to6Ny82MQq9z50c1nr3nbg/9YwdgMrznj+rNnmrWu6dGTRPnHO0fPynvPDCem65btO8va+IH8xsh3Ou2e865sKcjmEB99O9R3jX11r5yOsupS5WcmIWq//k2azu4XFm+q+quiwyJUyVHT9cuHRKwGqoKqWuooTwfBxay2agb//JZ39O9qwNdcKyy7zgNdl4X5cMzFGZfAyPp9lzsJ9HD/Tx6IE+dh/o4+mjg8dD2pLKkuOzaJNfl9fMY0iblB73ZignF6g9sANcxpucaXrBiSU2lm04p//fZzt+Le5AdjrOwXDXDLNr7d4MW3/HyWfthEu8HrWZZtbqEgU53fgPv7GDh/f38cubXzbnry1STBTIFqa+4Qm2/N2PTzp0WBIOnRSmpvdlTQauJZWlp51VKzrOLcjwNVvD42n2Hurn0Y4+dh/sZ/eBPp46Okgm92dfH5sW0hprWDHfIW20H/b98kT/Wede7/HyOCRfdKIHLX5+Xn+Wsx2/Fu4hy3Nl5h2Lji3xPslMlx73PglNnVGbvHVsh9G+k/cvj08La1Nm2qobz+r03JZEnLsfPczB3hFW1Gq1ahEJlpqKKHf94ZUMjE14s1qVZVSXR4J/JuBMFuLPlIeKkghbmuJsaTqxmOvIeIa9h71w9miHN5v2i6ePHQ9p8eMhrZr1K7yg1lhXXri/H2XVcPGrvBt4y6i0PXBiBm3P97zHa1bDb38YLnvrnL69AtnZipTkFh28YObtIz3eTNr0sHbwIe/K95PXDQPvGm61q059skF53Yz/mFsm1yNr7+YNG1fO4Q8n8y6b8fpSRnu9RSsr6otjBW2RArus0f9mavFHeUmYzavrTlpPc3Qiw95DuZB2oI9HD/TzL/c/e3wWta4i6vWjTZ44UMiQVrUMNrzNu01eA3RyeY0CXCVAgaxQyuu824qNz9+WSXsnFMx0KHTvD2D42Mn7l9bkZtRODmqXVDdRW7rwAplzjkzWMZFxTGSzTKSzpLMO57zTtiMhIxzOfQ0ZkVCIkFEcn6qdg7EBL5CPdHvXaBvp8W7D3dMe6z7x2PQZVcwLZZVLveu5xRpmvj/5dYEs5yIii1tZNMym1XVsmhbSHj884PWj5WbS/vWBEyGttiJ6fAZtMqStis9xSDPz1qdbciGk3jN3rzuFApkfwpETlwWZvoAheP+hT59d690HnU/Ak/ccP4snAuw0o/vhOuhogNJKKKmE0ipcSYxsSRXZaIxMpJJ0NEYmEmMiUsFEOMZ4OMZ4uIKxcAXj4RijoXImXJh0JstEJstExpHOZplInxyKxjNZ0hlHOpNlPPd1IpNlIjt53+We7+13fP/syfunj4et3Lbc60++99k4EdCMSDh00vdegAud+D50cqALh4xI+OTHy0hTxQA12X4q3SBVrp/KbD+VmX4qMgPEMv1UZPooT/dRnhmgfKKP0nQfYZc+ZY0TkUomSmuZKKklU1pLuno9maW1ZErryJbX4kprqXAjVEx0UTbWRXikCwaPepdHGew8cXbWdKU13kWBY0u9w+yVS59/vzIX5kqefyq+iEixKouG2biqlo2rao8/NjqR4YnJkJabTfvyL549/v9HTXmU9SurTwppq+MVxfHB/RQUyIpRaRUsW+/dpstmYfDw8aC2fedO9j37JPFjY5QzQoU7SIUbJWYjxBglxghlNpuVoWHURVCr8aoAAAsfSURBVBmknCFXxhDlDFLGkCtjkHIGXTlpypignMHc9lErZyxUwUgoF+xCFYznAl8mVE40GiISChENG9FwiPKSECW5YBQJGyVh72s0HCKaC1DRSIhoyHssEj7x3EjYMIyMc2QyXnjLZN1JX9OZ7LTHvO8z6TQlEwOUpvson+ijLN1Lebqf8kw/sfE+KtL9xLJewKrK9lPpBqjKDlDO6Cl/V2MuSi+V9FDJ/mwVPSyh1zXRSxU9rpJeKul1lfS4SnqootdV0keMNBEYPOXLTpEEoKIkTE151LstidJQmmZlySDLwgM0hPr///buPUausg7j+PeZ3dnd7nYv7W5be8ECipcWLXhBQmNCMCZ4xwRFFKLGxJhgotHES6LBy/9qTIxClACBeIfEGBNFJBAIdwQsF6Vy0SLSIu10Z3e6O5eff5zT7W7ZXrad2bN7zvNJJjtz5uzM+25nf332Pee8L6NUGGntY6i5l4H6Xvqm/kf5xScoPXMHmj0D+WzdK2aFtzXz3z848naEw+VmZlnqK3ex7ZQRts0KaVONJv/4b3XO1Z3X3PnMTEgb7OvmzA3JBQMHg9rm1f3ZT26bciBbbkqlZHLboQ2w+TxefdqH+f1t/ySIucFmVpjpUYOBmKQvJulrTbIiavQ2a/S2Juht1ii3JuhpTlJuTFBuTDDSmGC0PkFXY4KuepVSfQ+l6SqqV9H0EUZomumtnj5WKRmJ6VmZjNx1DUJXMno3s23m62Dydea5w/cZTJbLiEimInnFIb9ZhwQPzHNI8ECFZNLFeaiUhI6B1cnX/tclF2D0r4YVI7PuH3w+ud/b0886YF36Mq15guCc0Ng8wvZWMlp48HG92aI61aBSq7Nvsk6lVp+5v79W56m9wf21MpXaIAfqA8D6I35UVvXB5r5JNvdW2dhdZV33OGuVhLjh2MfQ1F4Gqk/T96/7KU+9jGKeySBL3YcOkx7rsGn/2LJbO87M8qO3u4s3bRqec17idKPFP14cnxPSrr3rWaabSb0b7O1m68ahOVNwnDo6kElIc/Vc5tYPr+C7F80zktYprVYSiqarMFWF6fH062GPp8bn32fipbn7tOrHfk9I1piL1tH37xmE/lWHwtOqzbMC1aq54ao/Pcevd7gt88eVSqJn5hd4cU7GP1Bvsr82N7TN3K/VZ57bNznNv2t1KrOeP/ywsGixiipjqrCutJ9NPVU2lseTkbjWOKPj+xipPM9QcwcD9Zfpilf+OwSC/tXoeA6bDqyF8iLP4G1mhdPTXZq5CODSdNvBkLZjVki77u7nmG4cCmlbNgzNrDZw5sZhTluEkOZAZgtTKiWXBrdrXrXG1NGD3XQa3KbGk9GseUar6F8NfSPJla8F0lfuoq/cxdoFLk0SkaylN98IXBLmpqnU6vy91uDeyelD26eSfVoRDFJjTBXGqDCq/YypwhpVWDteYV2tytr/7WFUOxlpVVgRk/O2Y+pLT9E7tLYdPwozs+M2O6R9LN1Wb84NaX97fj/X33MopA30dLH14IUDm4Y49/RR1g+3d7opBzLLVndvchsYzbolhSGJ/p5kseCFzl/XagXV6cac0bbZwe7Z2jSPHrb9wGSV7tpLrJh+mVFVGNN+xqjwufIwvjbUzJaCcleJrRuG2bphmEvenmyrN1s89WKVHc9X2PGfJKjdeO9zTN3V4lsf2MKntp/W1jY4kJnZcSuVxFBfmaG+Mqcs8HsbzRbjBxrsSwPbyr5ijWia2fJS7iqxZcMQWzYM8dG04jWaLXbuqTI60P4/Jx3IzGxRdHeVWDXQw6o2LYBsZrbYurtKvOFV7V8KEWDxVsM2MzMzs3k5kJmZmZllzIHMzMzMLGMOZGZmZmYZcyAzMzMzy1jHApmkayTtlrTjCM9L0g8l7ZT0qKS3dKotZmZmZktZJ0fIrgUuPMrz7wHOSG+fBX7cwbaYmZmZLVkdC2QRcQfw8lF2+RBwfSTuAUYkHXmlZDMzM7OcyvIcso3Av2c93pVuMzMzMyuUZXFSv6TPSnpA0gN79uzJujlmZmZmbZXl0knPw5zl8Dal214hIq4GrgaQtEfScwt4nzHgpRNt5DJRhD6C+5knC+3j5k41ZLE9+OCDLy2ghhXhswDF6GcR+gju53yOq35lGch+B3xe0i+AdwCViHjhWN8UEWsW8iaSHoiIt51gG5eFIvQR3M88KUIfj2QhNawoP6ci9LMIfQT382R0LJBJ+jlwPjAmaRdwJVAGiIifAH8A3gvsBCaBT3eqLWZmZmZLWccCWURceoznA7iiU+9vZmZmtlwsi5P6T9LVWTdgERShj+B+5kkR+tgORfk5FaGfRegjuJ8nTMlAlZmZmZllpQgjZGZmZmZLWm4DmaQLJf09XSvza1m3pxOOtV5oXkg6RdJtkh6X9JikL2TdpnaT1CfpPkmPpH38dtZt6iRJXZL+Kun3WbdlqXINy4ci1C8oVg3rVP3KZSCT1AX8iGS9zC3ApZK2ZNuqjriWo68XmhcN4MsRsQU4F7gih/+eU8AFEbENOAu4UNK5Gbepk74APJF1I5Yq17BcKUL9gmLVsI7Ur1wGMuAcYGdEPB0R08AvSNbOzJXjWC80FyLihYh4KL0/TvKLkKtlttI1Xavpw3J6y+UJnpI2Ae8Dfpp1W5Yw17CcKEL9guLUsE7Wr7wGMq+TmVOSTgXOBu7NtiXtlw6DPwzsBm6JiNz1MfUD4CtAK+uGLGGuYTmU5/oFhalhHatfeQ1klkOSVgK/Bb4YEfuzbk+7RUQzIs4iWUbsHElnZt2mdpP0fmB3RDyYdVvMFlPe6xfkv4Z1un7lNZAd9zqZtjxIKpMUsxsj4qas29NJEbEPuI18nluzHfigpGdJDsNdIOmGbJu0JLmG5UiR6hfkuoZ1tH7lNZDdD5wh6TRJPcDHSNbOtGVIkoCfAU9ExPeybk8nSFojaSS9vwJ4N/Bktq1qv4j4ekRsiohTSX4v/xIRl2XcrKXINSwnilC/oBg1rNP1K5eBLCIawOeBP5KcQPmriHgs21a1X7pe6N3A6yXtkvSZrNvUIduBy0n+Gnk4vb0360a12XrgNkmPkvxnfEtEeEqIgnINy5Ui1C9wDTtpnqnfzMzMLGO5HCEzMzMzW04cyMzMzMwy5kBmZmZmljEHMjMzM7OMOZCZmZmZZcyBzHJD0vmSfJm1mS1LrmHF5kBmZmZmljEHMlt0ki6TdF86QeJV6YK0VUnfl/SYpFslrUn3PUvSPZIelXSzpFXp9tdK+rOkRyQ9JOk16cuvlPQbSU9KujGdJdvMrG1cw6wTHMhsUUl6I3AJsD1dhLYJfAIYAB6IiK3A7cCV6bdcD3w1It4M/G3W9huBH0XENuA84IV0+9nAF4EtwOkks2SbmbWFa5h1SnfWDbDCeRfwVuD+9A+/FcBuoAX8Mt3nBuAmScPASETcnm6/Dvi1pEFgY0TcDBARBwDS17svInaljx8GTgXu7Hy3zKwgXMOsIxzIbLEJuC4ivj5no/TNw/Y70TW9pmbdb+LPuJm1l2uYdYQPWdpiuxW4WNJaAEmrJW0m+SxenO7zceDOiKgAeyW9M91+OXB7RIwDuyRdlL5Gr6T+Re2FmRWVa5h1hJO3LaqIeFzSN4A/SSoBdeAKYAI4J31uN8k5GgCfBH6SFqungU+n2y8HrpL0nfQ1PrKI3TCzgnINs05RxImOqpq1j6RqRKzMuh1mZifCNcxOlg9ZmpmZmWXMI2RmZmZmGfMImZmZmVnGHMjMzMzMMuZAZmZmZpYxBzIzMzOzjDmQmZmZmWXMgczMzMwsY/8HaupRcFCV9dQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot\n",
    "import matplotlib.pyplot as plt\n",
    "nrows = 1\n",
    "ncols = 2\n",
    "fig = plt.figure(figsize=(10, 5))\n",
    "\n",
    "for idx, key in enumerate([\"loss\", \"rmse\"]):\n",
    "    ax = fig.add_subplot(nrows, ncols, idx+1)\n",
    "    plt.plot(history.history[key])\n",
    "    plt.plot(history.history[\"val_{}\".format(key)])\n",
    "    plt.title(\"model {}\".format(key))\n",
    "    plt.ylabel(key)\n",
    "    plt.xlabel(\"epoch\")\n",
    "    plt.legend([\"train\", \"validation\"], loc=\"upper left\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "INFO:tensorflow:Assets written to: babyweight_trained/20191119050541/assets\n",
      "Exported trained model to babyweight_trained/20191119050541\n"
     ]
    }
   ],
   "source": [
    "OUTPUT_DIR = \"babyweight_trained\"\n",
    "shutil.rmtree(OUTPUT_DIR, ignore_errors=True)\n",
    "EXPORT_PATH = os.path.join(\n",
    "    OUTPUT_DIR, datetime.datetime.now().strftime(\"%Y%m%d%H%M%S\"))\n",
    "tf.saved_model.save(\n",
    "    obj=model, export_dir=EXPORT_PATH)  # with default serving function\n",
    "print(\"Exported trained model to {}\".format(EXPORT_PATH))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "assets\tsaved_model.pb\tvariables\n"
     ]
    }
   ],
   "source": [
    "!ls $EXPORT_PATH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2020 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
